[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:41 dist-test-slave-f0tn krb5kdc[24337](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:41 dist-test-slave-f0tn krb5kdc[24337](info): set up 2 sockets
Aug 08 22:59:41 dist-test-slave-f0tn krb5kdc[24337](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 22:59:42.367048 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 22:59:42 dist-test-slave-f0tn krb5kdc[24337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 22:59:42.478175 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:46481
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:46169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:46481
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:42.566160 24356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:42.566370 24356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:42.566392 24356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:42.567498 24356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:42.567533 24356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:42.567546 24356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:42.567559 24356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:42.567571 24356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:42.569252 24356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:46169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:46481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:46481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:42.569624 24356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:42.569842 24356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:42.575809 24365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:42.575809 24371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:42.576550 24364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:42.577312 24356 server_base.cc:1034] running on GCE node
I20260808 22:59:42.577637 24356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:42.577991 24356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:42.579123 24356 hybrid_clock.cc:648] HybridClock initialized: now 1786229982579082 us; error 55 us; skew 500 ppm
Aug 08 22:59:42 dist-test-slave-f0tn krb5kdc[24337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229982, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:42.581602 24356 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 22:59:42.582263 24356 webserver.cc:460] Webserver started at http://127.23.194.62:38561/ using document root <none> and password file <none>
I20260808 22:59:42.582485 24356 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:42.582528 24356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:42.582657 24356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:42.583760 24356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "70bee891711f400ab2a99b124b3e1c4f"
format_stamp: "Formatted at 2026-08-08 22:59:42 on dist-test-slave-f0tn"
server_key: "927451d3aa818be95d2901d56023cafe"
server_key_iv: "427d021b0d3f6d8d611d42d0f21d87f3"
server_key_version: "encryptionkey@0"
I20260808 22:59:42.584091 24356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "70bee891711f400ab2a99b124b3e1c4f"
format_stamp: "Formatted at 2026-08-08 22:59:42 on dist-test-slave-f0tn"
server_key: "927451d3aa818be95d2901d56023cafe"
server_key_iv: "427d021b0d3f6d8d611d42d0f21d87f3"
server_key_version: "encryptionkey@0"
I20260808 22:59:42.585585 24356 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:59:42.586272 24379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:42.586460 24356 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 22:59:42.586550 24356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "70bee891711f400ab2a99b124b3e1c4f"
format_stamp: "Formatted at 2026-08-08 22:59:42 on dist-test-slave-f0tn"
server_key: "927451d3aa818be95d2901d56023cafe"
server_key_iv: "427d021b0d3f6d8d611d42d0f21d87f3"
server_key_version: "encryptionkey@0"
I20260808 22:59:42.586611 24356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:42.597864 24356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:42.599784 24356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:42.599970 24356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:42.604858 24356 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:46481
I20260808 22:59:42.604933 24465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:46481 every 8 connection(s)
I20260808 22:59:42.605634 24356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:42.606927 24467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:42.610208 24467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f: Bootstrap starting.
I20260808 22:59:42.611121 24467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:42.611510 24467 log.cc:824] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:42.612313 24467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f: No bootstrap required, opened a new log
I20260808 22:59:42.612772 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 24356
I20260808 22:59:42.612890 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 22:59:42.613698 24328 external_mini_cluster.cc:1428] Setting key b85e7bf980aba1c377032bff4a09e0d4
I20260808 22:59:42.614825 24467 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70bee891711f400ab2a99b124b3e1c4f" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46481 } }
I20260808 22:59:42.615032 24467 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:42.615144 24467 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70bee891711f400ab2a99b124b3e1c4f, State: Initialized, Role: FOLLOWER
I20260808 22:59:42.615371 24467 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70bee891711f400ab2a99b124b3e1c4f" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46481 } }
I20260808 22:59:42.615558 24467 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:42.615625 24467 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:42.615679 24467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:42.616683 24467 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70bee891711f400ab2a99b124b3e1c4f" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46481 } }
I20260808 22:59:42.616860 24467 leader_election.cc:302] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70bee891711f400ab2a99b124b3e1c4f; no voters:
I20260808 22:59:42.617064 24467 leader_election.cc:288] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:42.617100 24475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:42.617252 24475 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f [term 1 LEADER]: Becoming Leader. State: Replica: 70bee891711f400ab2a99b124b3e1c4f, State: Running, Role: LEADER
I20260808 22:59:42.617473 24475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70bee891711f400ab2a99b124b3e1c4f" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46481 } }
I20260808 22:59:42.617645 24467 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:42.618046 24476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70bee891711f400ab2a99b124b3e1c4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70bee891711f400ab2a99b124b3e1c4f" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46481 } } }
I20260808 22:59:42.618290 24476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:42.618218 24477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70bee891711f400ab2a99b124b3e1c4f. Latest consensus state: current_term: 1 leader_uuid: "70bee891711f400ab2a99b124b3e1c4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70bee891711f400ab2a99b124b3e1c4f" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46481 } } }
I20260808 22:59:42.618593 24477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:42.619197 24486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:42.620375 24486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:42.622786 24486 catalog_manager.cc:1347] Generated new cluster ID: 1b79af14d7634b0b8d97d041fbdccf1e
I20260808 22:59:42.622835 24486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 22:59:42 dist-test-slave-f0tn krb5kdc[24337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:42.631750 24471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:42.615069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42528 (local address 127.23.194.62:46481)
0808 22:59:42.615452 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:42.615463 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:42.615616 (+ 153us) server_negotiation.cc:407] Connection header received
0808 22:59:42.617632 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:42.617647 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:42.617756 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:42.617890 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:42.620233 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:42.621223 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:42.622634 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:42.623244 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:42.623345 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:42.627774 (+ 4429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:42.627778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:42.627779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:42.627790 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:42.629752 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:42.630209 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:42.630212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:42.630212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:42.630334 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:42.630579 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:42.630580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:42.630580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:42.630914 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:42.631084 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:42.631319 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 22:59:42.631507 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":100,"threads_started":1}
I20260808 22:59:42.635696 24486 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:42.636221 24486 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:42.642372 24486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70bee891711f400ab2a99b124b3e1c4f: Generated new TSK 0
I20260808 22:59:42.642561 24486 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
I20260808 22:59:42.671856 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:46481
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:46169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:42.772287 24504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:42.772500 24504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:42.772533 24504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:42.773717 24504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:42.773759 24504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:42.773795 24504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 22:59:42.775372 24504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:46169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:46481
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:42.775657 24504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:42.775859 24504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:42.782502 24514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:42.782536 24515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:42.782604 24504 server_base.cc:1034] running on GCE node
W20260808 22:59:42.782708 24517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:42.782933 24504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:42.783175 24504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:42.784320 24504 hybrid_clock.cc:648] HybridClock initialized: now 1786229982784304 us; error 33 us; skew 500 ppm
Aug 08 22:59:42 dist-test-slave-f0tn krb5kdc[24337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229982, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:42.787416 24504 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 22:59:42.787904 24504 webserver.cc:460] Webserver started at http://127.23.194.1:36349/ using document root <none> and password file <none>
I20260808 22:59:42.788133 24504 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:42.788178 24504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:42.788301 24504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:42.789542 24504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "2fbab51e4ff24a4ebe5338fcaa270561"
format_stamp: "Formatted at 2026-08-08 22:59:42 on dist-test-slave-f0tn"
server_key: "1106cbdbc5e5bf526cbc781393f82664"
server_key_iv: "0e4e9f37595bcb41a158b9061f7d98e0"
server_key_version: "encryptionkey@0"
I20260808 22:59:42.789911 24504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "2fbab51e4ff24a4ebe5338fcaa270561"
format_stamp: "Formatted at 2026-08-08 22:59:42 on dist-test-slave-f0tn"
server_key: "1106cbdbc5e5bf526cbc781393f82664"
server_key_iv: "0e4e9f37595bcb41a158b9061f7d98e0"
server_key_version: "encryptionkey@0"
I20260808 22:59:42.791507 24504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 22:59:42.792294 24525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:42.792482 24504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:59:42.792594 24504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "2fbab51e4ff24a4ebe5338fcaa270561"
format_stamp: "Formatted at 2026-08-08 22:59:42 on dist-test-slave-f0tn"
server_key: "1106cbdbc5e5bf526cbc781393f82664"
server_key_iv: "0e4e9f37595bcb41a158b9061f7d98e0"
server_key_version: "encryptionkey@0"
I20260808 22:59:42.792675 24504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:42.826507 24504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:42.828341 24504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:42.828454 24504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:42.828713 24504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:59:42.829074 24504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:59:42.829157 24504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:42.829185 24504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:59:42.829200 24504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:42.841337 24504 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:38775
I20260808 22:59:42.841472 24716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:38775 every 8 connection(s)
I20260808 22:59:42.841799 24504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 22:59:42.847211 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 24504
I20260808 22:59:42.847301 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 22:59:42.847440 24328 external_mini_cluster.cc:1428] Setting key 3b2ce1f1efcf957846965239b9d20c4e
Aug 08 22:59:42 dist-test-slave-f0tn krb5kdc[24337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229982, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:42.851994 24471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:42.842783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:54473 (local address 127.23.194.62:46481)
0808 22:59:42.842848 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:42.842850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:42.843166 (+ 316us) server_negotiation.cc:407] Connection header received
0808 22:59:42.843741 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:42.843742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:42.843755 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:42.843807 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:42.845014 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:42.845621 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:42.846667 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:42.846869 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:42.846926 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:42.849218 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:42.849219 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:42.849220 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:42.849229 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:42.850670 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:42.851050 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:42.851051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:42.851052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:42.851104 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:42.851326 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:42.851327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:42.851327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:42.851434 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:42.851529 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:42.851815 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 22:59:42.851904 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:59:42.852208 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:42.842753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:46481 (local address 127.23.194.1:54473)
0808 22:59:42.843004 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:42.843042 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:42.843652 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:42.843875 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:42.843878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:42.844649 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:42.844916 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:42.844921 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:42.845694 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:42.845694 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:42.846584 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:42.846587 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:42.846932 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:42.846932 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:42.847026 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:42.847603 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:42.847612 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:42.849132 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:42.850745 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:42.850752 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:42.850755 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:42.850972 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:42.851161 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:42.851162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:42.851162 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:42.851269 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:42.851542 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:42.851549 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:42.851692 (+ 143us) client_negotiation.cc:769] Sending connection context
0808 22:59:42.851781 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 22:59:42.851935 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":155,"threads_started":1}
I20260808 22:59:42.852794 24717 heartbeater.cc:347] Connected to a master server at 127.23.194.62:46481
I20260808 22:59:42.852921 24717 heartbeater.cc:464] Registering TS with master...
I20260808 22:59:42.853135 24717 heartbeater.cc:511] Master 127.23.194.62:46481 requested a full tablet report, sending...
I20260808 22:59:42.854125 24411 ts_manager.cc:194] Registered new tserver with Master: 2fbab51e4ff24a4ebe5338fcaa270561 (127.23.194.1:38775)
I20260808 22:59:42.854897 24411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:54473
WARNING: no policy specified for kudu/127.23.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
I20260808 22:59:42.873814 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:46481
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:46169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:42.986943 24725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:42.987145 24725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:42.987167 24725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:42.988665 24725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:42.988708 24725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:42.988735 24725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 22:59:42.990432 24725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:46169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:46481
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:42.990684 24725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:42.990906 24725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:42.997113 24735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:42.997150 24737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:42.997169 24725 server_base.cc:1034] running on GCE node
W20260808 22:59:42.997149 24741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:42.997538 24725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:42.997782 24725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:42.998929 24725 hybrid_clock.cc:648] HybridClock initialized: now 1786229982998910 us; error 29 us; skew 500 ppm
Aug 08 22:59:42 dist-test-slave-f0tn krb5kdc[24337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229982, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:43.002053 24725 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 22:59:43.002516 24725 webserver.cc:460] Webserver started at http://127.23.194.2:40695/ using document root <none> and password file <none>
I20260808 22:59:43.002697 24725 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:43.002737 24725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:43.002828 24725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:43.003968 24725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "f482c001edf540a9b108ab4d4e9c627a"
format_stamp: "Formatted at 2026-08-08 22:59:42 on dist-test-slave-f0tn"
server_key: "f808a051fcbf2e5465b1a3d1b1dd9345"
server_key_iv: "f0c67e446d31091bfa6b9a0551910c0d"
server_key_version: "encryptionkey@0"
I20260808 22:59:43.004308 24725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "f482c001edf540a9b108ab4d4e9c627a"
format_stamp: "Formatted at 2026-08-08 22:59:42 on dist-test-slave-f0tn"
server_key: "f808a051fcbf2e5465b1a3d1b1dd9345"
server_key_iv: "f0c67e446d31091bfa6b9a0551910c0d"
server_key_version: "encryptionkey@0"
I20260808 22:59:43.005599 24725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:59:43.006357 24752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:43.006520 24725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 22:59:43.006608 24725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "f482c001edf540a9b108ab4d4e9c627a"
format_stamp: "Formatted at 2026-08-08 22:59:42 on dist-test-slave-f0tn"
server_key: "f808a051fcbf2e5465b1a3d1b1dd9345"
server_key_iv: "f0c67e446d31091bfa6b9a0551910c0d"
server_key_version: "encryptionkey@0"
I20260808 22:59:43.006664 24725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:43.019309 24725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:43.021091 24725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:43.021211 24725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:43.021412 24725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:59:43.021688 24725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:59:43.021719 24725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:43.021761 24725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:59:43.021777 24725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:43.034919 24725 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:36447
I20260808 22:59:43.035059 24916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:36447 every 8 connection(s)
I20260808 22:59:43.035343 24725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 22:59:43.038717 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 24725
I20260808 22:59:43.038818 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 22:59:43.039001 24328 external_mini_cluster.cc:1428] Setting key d2228a7bd695047e4f9b89fb9bf7b96f
Aug 08 22:59:43 dist-test-slave-f0tn krb5kdc[24337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229982, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:43.046041 24471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:43.036117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:56873 (local address 127.23.194.62:46481)
0808 22:59:43.036199 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:43.036201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:43.036453 (+ 252us) server_negotiation.cc:407] Connection header received
0808 22:59:43.037207 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:43.037208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:43.037220 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:43.037284 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:43.038394 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:43.039101 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:43.040458 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:43.040730 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:43.040807 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:43.043359 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:43.043360 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:43.043361 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:43.043367 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:43.044676 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:43.045136 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:43.045136 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:43.045137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:43.045185 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:43.045461 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:43.045462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:43.045462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:43.045560 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:43.045610 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:43.045913 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 22:59:43.045978 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 22:59:43.046342 24925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:43.036084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:46481 (local address 127.23.194.2:56873)
0808 22:59:43.036323 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:43.036364 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:43.037112 (+ 748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:43.037359 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:43.037363 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:43.038051 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:43.038299 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:43.038304 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:43.039206 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:43.039207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:43.040353 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:43.040356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:43.040806 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:43.040807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:43.040933 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:43.041532 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:43.041541 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:43.043271 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:43.044750 (+ 1479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:43.044757 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:43.044759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:43.045056 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:43.045240 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:43.045241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:43.045241 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:43.045396 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:43.045620 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:43.045626 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:43.045785 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 22:59:43.045894 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 22:59:43.046055 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":120,"threads_started":1}
I20260808 22:59:43.046923 24922 heartbeater.cc:347] Connected to a master server at 127.23.194.62:46481
I20260808 22:59:43.047029 24922 heartbeater.cc:464] Registering TS with master...
I20260808 22:59:43.047238 24922 heartbeater.cc:511] Master 127.23.194.62:46481 requested a full tablet report, sending...
I20260808 22:59:43.047600 24411 ts_manager.cc:194] Registered new tserver with Master: f482c001edf540a9b108ab4d4e9c627a (127.23.194.2:36447)
I20260808 22:59:43.047955 24411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:56873
WARNING: no policy specified for kudu/127.23.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
I20260808 22:59:43.065749 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:46481
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:46169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:43.181661 24929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:43.181875 24929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:43.181897 24929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:43.183041 24929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:43.183084 24929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:43.183117 24929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 22:59:43.184716 24929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:46169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:46481
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:43.184929 24929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:43.185143 24929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:43.190793 24935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:43.190840 24934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:43.190840 24937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:43.191113 24929 server_base.cc:1034] running on GCE node
I20260808 22:59:43.191291 24929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:43.191509 24929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:43.192674 24929 hybrid_clock.cc:648] HybridClock initialized: now 1786229983192659 us; error 29 us; skew 500 ppm
Aug 08 22:59:43 dist-test-slave-f0tn krb5kdc[24337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229983, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:43.195425 24929 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 22:59:43.195811 24929 webserver.cc:460] Webserver started at http://127.23.194.3:45311/ using document root <none> and password file <none>
I20260808 22:59:43.195999 24929 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:43.196084 24929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:43.196210 24929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:43.197465 24929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "5207b1bbc5b441159db4c5c48f0e25c5"
format_stamp: "Formatted at 2026-08-08 22:59:43 on dist-test-slave-f0tn"
server_key: "afef199dca4b11e5d18f5092c3d38f40"
server_key_iv: "2c1e5acf99e2e745d5b36657496e6421"
server_key_version: "encryptionkey@0"
I20260808 22:59:43.197836 24929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "5207b1bbc5b441159db4c5c48f0e25c5"
format_stamp: "Formatted at 2026-08-08 22:59:43 on dist-test-slave-f0tn"
server_key: "afef199dca4b11e5d18f5092c3d38f40"
server_key_iv: "2c1e5acf99e2e745d5b36657496e6421"
server_key_version: "encryptionkey@0"
I20260808 22:59:43.199199 24929 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:59:43.199947 24947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:43.200094 24929 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 22:59:43.200179 24929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "5207b1bbc5b441159db4c5c48f0e25c5"
format_stamp: "Formatted at 2026-08-08 22:59:43 on dist-test-slave-f0tn"
server_key: "afef199dca4b11e5d18f5092c3d38f40"
server_key_iv: "2c1e5acf99e2e745d5b36657496e6421"
server_key_version: "encryptionkey@0"
I20260808 22:59:43.200235 24929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:43.235870 24929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:43.238127 24929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:43.238314 24929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:43.238741 24929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:59:43.239113 24929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:59:43.239152 24929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:43.239183 24929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:59:43.239207 24929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:43.252018 24929 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:42415
I20260808 22:59:43.252070 25128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:42415 every 8 connection(s)
I20260808 22:59:43.252414 24929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
Aug 08 22:59:43 dist-test-slave-f0tn krb5kdc[24337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229983, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:43.261471 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 24929
I20260808 22:59:43.261575 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 22:59:43.261746 24328 external_mini_cluster.cc:1428] Setting key 85c533b7e0613bcffba57ab8e9f9a56a
I20260808 22:59:43.262712 24471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:43.253218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:52503 (local address 127.23.194.62:46481)
0808 22:59:43.253271 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:43.253273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:43.253579 (+ 306us) server_negotiation.cc:407] Connection header received
0808 22:59:43.254194 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:43.254196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:43.254209 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:43.254262 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:43.255309 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:43.256084 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:43.257192 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:43.257402 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:43.257469 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:43.259695 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:43.259697 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:43.259697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:43.259704 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:43.260823 (+ 1119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:43.261222 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:43.261223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:43.261223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:43.261269 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:43.262037 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:43.262038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:43.262039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:43.262187 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:43.262271 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:43.262558 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 22:59:43.262645 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 22:59:43.262928 25131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:43.253169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:46481 (local address 127.23.194.3:52503)
0808 22:59:43.253454 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:43.253493 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:43.254091 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:43.254339 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:43.254343 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:43.254945 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:43.255214 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:43.255219 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:43.256184 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:43.256186 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:43.257111 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:43.257114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:43.257475 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:43.257476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:43.257611 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:43.258157 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:43.258165 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:43.259587 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:43.260909 (+ 1322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:43.260916 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:43.260918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:43.261156 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:43.261862 (+ 706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:43.261863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:43.261863 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:43.261970 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:43.262262 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:43.262268 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:43.262420 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 22:59:43.262524 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 22:59:43.262709 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":139,"threads_started":1}
I20260808 22:59:43.263406 25129 heartbeater.cc:347] Connected to a master server at 127.23.194.62:46481
I20260808 22:59:43.263491 25129 heartbeater.cc:464] Registering TS with master...
I20260808 22:59:43.263638 25129 heartbeater.cc:511] Master 127.23.194.62:46481 requested a full tablet report, sending...
I20260808 22:59:43.263959 24411 ts_manager.cc:194] Registered new tserver with Master: 5207b1bbc5b441159db4c5c48f0e25c5 (127.23.194.3:42415)
I20260808 22:59:43.264290 24411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:52503
I20260808 22:59:43.274029 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 22:59:43 dist-test-slave-f0tn krb5kdc[24337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229983, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 22:59:43 dist-test-slave-f0tn krb5kdc[24337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229983, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.194.1@KRBTEST.COM
I20260808 22:59:43.297353 25138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:43.290395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41334 (local address 127.23.194.1:38775)
0808 22:59:43.290576 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:43.290580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:43.290590 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:59:43.290616 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:43.290618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:43.290631 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:43.290699 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:43.291221 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:43.291911 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:43.292684 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:43.292981 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:43.293029 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:43.295133 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:43.295142 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:43.295142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:43.295150 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:43.296461 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:43.296764 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:43.296766 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:43.296767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:43.296822 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:43.296953 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:43.296954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:43.296954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:43.297071 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:43.297185 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:43.297218 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 22:59:43.297278 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":94,"threads_started":1}
W20260808 22:59:43.297861 24579 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41334
Aug 08 22:59:43 dist-test-slave-f0tn krb5kdc[24337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 22:59:43 dist-test-slave-f0tn krb5kdc[24337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.1@KRBTEST.COM
I20260808 22:59:43.318732 25138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:43.312376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41336 (local address 127.23.194.1:38775)
0808 22:59:43.312440 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:43.312442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:43.312491 (+ 49us) server_negotiation.cc:407] Connection header received
0808 22:59:43.312549 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:43.312550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:43.312559 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:43.312595 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:43.313130 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:43.313727 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:43.314384 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:43.314603 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:43.314648 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:43.316575 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:43.316577 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:43.316577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:43.316583 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:43.317743 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:43.318047 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:43.318048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:43.318048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:43.318098 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:43.318258 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:43.318258 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:43.318259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:43.318349 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:43.318502 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:43.318560 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 22:59:43.318639 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:59:43.319185 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 24504
I20260808 22:59:43.334929 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 24725
I20260808 22:59:43.351084 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 24929
I20260808 22:59:43.383714 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 24356
2026-08-08T22:59:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2178 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:43 dist-test-slave-f0tn krb5kdc[25150](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:43 dist-test-slave-f0tn krb5kdc[25150](info): set up 2 sockets
Aug 08 22:59:43 dist-test-slave-f0tn krb5kdc[25150](info): commencing operation
krb5kdc: starting...
W20260808 22:59:44.562006 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260808 22:59:44.702517 24328 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.140s user 0.001s sys 0.001s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260808 22:59:44.935278 24328 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.233s user 0.001s sys 0.001s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260808 22:59:45.073356 24328 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.138s user 0.003s sys 0.000s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/test-user.keytab.
W20260808 22:59:45.611838 24328 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.538s user 0.002s sys 0.000s
Aug 08 22:59:45 dist-test-slave-f0tn krb5kdc[25150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
W20260808 22:59:45.958069 24328 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.23.194.62: real 0.325s user 0.000s sys 0.003s
I20260808 22:59:45.958415 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:35143
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:34301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:35143
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:46.040963 25168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:46.041190 25168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:46.041224 25168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:46.042461 25168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:46.042510 25168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:46.042528 25168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:46.042546 25168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:46.042560 25168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:46.044054 25168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:34301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:35143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:35143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:46.044306 25168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:46.044507 25168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:46.049198 25178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:46.049221 25174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:46.049319 25168 server_base.cc:1034] running on GCE node
W20260808 22:59:46.049219 25175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:46.049638 25168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:46.049918 25168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:46.051060 25168 hybrid_clock.cc:648] HybridClock initialized: now 1786229986051053 us; error 36 us; skew 500 ppm
Aug 08 22:59:46 dist-test-slave-f0tn krb5kdc[25150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229986, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:46.053519 25168 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 22:59:46.053978 25168 webserver.cc:460] Webserver started at http://127.23.194.62:37553/ using document root <none> and password file <none>
I20260808 22:59:46.054186 25168 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:46.054234 25168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:46.054363 25168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:46.055397 25168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "e86e4c3074b14cb1afe7d2bd4b67d7f8"
format_stamp: "Formatted at 2026-08-08 22:59:46 on dist-test-slave-f0tn"
server_key: "e1c4deb856a939ff67758150011f6150"
server_key_iv: "63c6d607f0c6d39861fcbd36eb140d69"
server_key_version: "encryptionkey@0"
I20260808 22:59:46.055747 25168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "e86e4c3074b14cb1afe7d2bd4b67d7f8"
format_stamp: "Formatted at 2026-08-08 22:59:46 on dist-test-slave-f0tn"
server_key: "e1c4deb856a939ff67758150011f6150"
server_key_iv: "63c6d607f0c6d39861fcbd36eb140d69"
server_key_version: "encryptionkey@0"
I20260808 22:59:46.057010 25168 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:59:46.057789 25187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:46.057974 25168 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 22:59:46.058065 25168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "e86e4c3074b14cb1afe7d2bd4b67d7f8"
format_stamp: "Formatted at 2026-08-08 22:59:46 on dist-test-slave-f0tn"
server_key: "e1c4deb856a939ff67758150011f6150"
server_key_iv: "63c6d607f0c6d39861fcbd36eb140d69"
server_key_version: "encryptionkey@0"
I20260808 22:59:46.058135 25168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:46.077800 25168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:46.079838 25168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:46.079978 25168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:46.083897 25168 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:35143
I20260808 22:59:46.083946 25264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:35143 every 8 connection(s)
I20260808 22:59:46.084281 25168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:46.085150 25265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:46.087419 25265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8: Bootstrap starting.
I20260808 22:59:46.088001 25265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:46.088264 25265 log.cc:824] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:46.088940 25265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8: No bootstrap required, opened a new log
I20260808 22:59:46.091607 25265 raft_consensus.cc:348] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86e4c3074b14cb1afe7d2bd4b67d7f8" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35143 } }
I20260808 22:59:46.091754 25265 raft_consensus.cc:374] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:46.091777 25265 raft_consensus.cc:729] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e86e4c3074b14cb1afe7d2bd4b67d7f8, State: Initialized, Role: FOLLOWER
I20260808 22:59:46.091923 25265 consensus_queue.cc:260] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86e4c3074b14cb1afe7d2bd4b67d7f8" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35143 } }
I20260808 22:59:46.091986 25265 raft_consensus.cc:388] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:46.092010 25265 raft_consensus.cc:482] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:46.092057 25265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:46.092782 25265 raft_consensus.cc:504] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86e4c3074b14cb1afe7d2bd4b67d7f8" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35143 } }
I20260808 22:59:46.092872 25265 leader_election.cc:302] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e86e4c3074b14cb1afe7d2bd4b67d7f8; no voters:
I20260808 22:59:46.093019 25265 leader_election.cc:288] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:46.093060 25270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:46.093248 25270 raft_consensus.cc:686] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8 [term 1 LEADER]: Becoming Leader. State: Replica: e86e4c3074b14cb1afe7d2bd4b67d7f8, State: Running, Role: LEADER
I20260808 22:59:46.093247 25265 sys_catalog.cc:564] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:46.093433 25270 consensus_queue.cc:237] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86e4c3074b14cb1afe7d2bd4b67d7f8" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35143 } }
I20260808 22:59:46.093688 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 25168
I20260808 22:59:46.093778 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 22:59:46.093845 25271 sys_catalog.cc:455] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e86e4c3074b14cb1afe7d2bd4b67d7f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86e4c3074b14cb1afe7d2bd4b67d7f8" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35143 } } }
I20260808 22:59:46.093936 24328 external_mini_cluster.cc:1428] Setting key cbeef4927c8313d54d5fab7a2b354b7a
I20260808 22:59:46.093928 25271 sys_catalog.cc:458] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:46.093974 25272 sys_catalog.cc:455] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e86e4c3074b14cb1afe7d2bd4b67d7f8. Latest consensus state: current_term: 1 leader_uuid: "e86e4c3074b14cb1afe7d2bd4b67d7f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86e4c3074b14cb1afe7d2bd4b67d7f8" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35143 } } }
I20260808 22:59:46.094040 25272 sys_catalog.cc:458] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:46.095103 25279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:46.096007 25279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:46.098003 25279 catalog_manager.cc:1347] Generated new cluster ID: fb08c940c63d402894f6ea68cb2945a9
I20260808 22:59:46.098057 25279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 22:59:46 dist-test-slave-f0tn krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:46.106482 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.094402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54284 (local address 127.23.194.62:35143)
0808 22:59:46.094780 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:46.094790 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:46.094824 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:59:46.095095 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:46.095103 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:46.095171 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:46.095328 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:46.096203 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.097209 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.098135 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.098559 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.098616 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.103448 (+ 4832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:46.103452 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:46.103452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:46.103463 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:46.104974 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:46.105302 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:46.105305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:46.105305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:46.105434 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:46.105615 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:46.105616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:46.105616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:46.105850 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:46.106001 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:46.106073 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 22:59:46.106324 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":218,"threads_started":1}
I20260808 22:59:46.109800 25279 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:46.110265 25279 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:46.115244 25279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e86e4c3074b14cb1afe7d2bd4b67d7f8: Generated new TSK 0
I20260808 22:59:46.115403 25279 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
I20260808 22:59:46.142174 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35143
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:34301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:46.246376 25309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:46.246568 25309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:46.246590 25309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:46.247767 25309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:46.247809 25309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:46.247836 25309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 22:59:46.249326 25309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:34301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35143
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:46.249577 25309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:46.249804 25309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:59:46.254630 25309 server_base.cc:1034] running on GCE node
W20260808 22:59:46.254567 25328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:46.254637 25319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:46.254567 25317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:46.254971 25309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:46.255220 25309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:46.256354 25309 hybrid_clock.cc:648] HybridClock initialized: now 1786229986256348 us; error 27 us; skew 500 ppm
Aug 08 22:59:46 dist-test-slave-f0tn krb5kdc[25150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229986, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:46.258721 25309 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 22:59:46.259133 25309 webserver.cc:460] Webserver started at http://127.23.194.1:44817/ using document root <none> and password file <none>
I20260808 22:59:46.259349 25309 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:46.259393 25309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:46.259510 25309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:46.260479 25309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "bade055e33b443b7ac55d830efacb509"
format_stamp: "Formatted at 2026-08-08 22:59:46 on dist-test-slave-f0tn"
server_key: "0eb30066c8e5bf53c7eefa6f8958252a"
server_key_iv: "c23c6783769c2317de406601f88d7442"
server_key_version: "encryptionkey@0"
I20260808 22:59:46.260789 25309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "bade055e33b443b7ac55d830efacb509"
format_stamp: "Formatted at 2026-08-08 22:59:46 on dist-test-slave-f0tn"
server_key: "0eb30066c8e5bf53c7eefa6f8958252a"
server_key_iv: "c23c6783769c2317de406601f88d7442"
server_key_version: "encryptionkey@0"
I20260808 22:59:46.262178 25309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 22:59:46.262812 25336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:46.262948 25309 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:46.263025 25309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "bade055e33b443b7ac55d830efacb509"
format_stamp: "Formatted at 2026-08-08 22:59:46 on dist-test-slave-f0tn"
server_key: "0eb30066c8e5bf53c7eefa6f8958252a"
server_key_iv: "c23c6783769c2317de406601f88d7442"
server_key_version: "encryptionkey@0"
I20260808 22:59:46.263087 25309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:46.277040 25309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:46.278916 25309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:46.279042 25309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:46.279258 25309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:59:46.279559 25309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:59:46.279600 25309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:46.279634 25309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:59:46.279659 25309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:46.285226 25309 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:34039
I20260808 22:59:46.285287 25512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:34039 every 8 connection(s)
I20260808 22:59:46.285602 25309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 22:59:46.287773 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 25309
I20260808 22:59:46.287868 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 22:59:46.288028 24328 external_mini_cluster.cc:1428] Setting key 24992a4ce2cf9579edc4d045a3720f00
Aug 08 22:59:46 dist-test-slave-f0tn krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229986, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:46.295825 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.286283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:43853 (local address 127.23.194.62:35143)
0808 22:59:46.286350 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:46.286352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:46.286523 (+ 171us) server_negotiation.cc:407] Connection header received
0808 22:59:46.287510 (+ 987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:46.287511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:46.287522 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:46.287577 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:46.288404 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.288987 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.290468 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.290768 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.290830 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.292980 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:46.292981 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:46.292982 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:46.292989 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:46.294405 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:46.294883 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:46.294884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:46.294884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:46.294936 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:46.295194 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:46.295194 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:46.295195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:46.295292 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:46.295336 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:46.295636 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 22:59:46.295729 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:59:46.296072 25519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.286242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35143 (local address 127.23.194.1:43853)
0808 22:59:46.286396 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:46.286438 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:46.287424 (+ 986us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:46.287665 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:46.287670 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:46.288028 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:46.288306 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:46.288313 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.289077 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.289078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:46.290360 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:46.290365 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.290856 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.290857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:46.290970 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.291464 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:46.291472 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:46.292881 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:46.294478 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:46.294486 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:46.294489 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:46.294818 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:46.294991 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:46.294992 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:46.294992 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:46.295128 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:46.295345 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:46.295350 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:46.295505 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 22:59:46.295617 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 22:59:46.295782 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260808 22:59:46.296648 25514 heartbeater.cc:347] Connected to a master server at 127.23.194.62:35143
I20260808 22:59:46.296751 25514 heartbeater.cc:464] Registering TS with master...
I20260808 22:59:46.296947 25514 heartbeater.cc:511] Master 127.23.194.62:35143 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.23.194.2@KRBTEST.COM; defaulting to no policy
I20260808 22:59:46.297647 25214 ts_manager.cc:194] Registered new tserver with Master: bade055e33b443b7ac55d830efacb509 (127.23.194.1:34039)
I20260808 22:59:46.298401 25214 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:43853
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
I20260808 22:59:46.311823 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35143
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:34301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:46.391350 25522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:46.391547 25522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:46.391570 25522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:46.392697 25522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:46.392738 25522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:46.392763 25522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 22:59:46.394311 25522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:34301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35143
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:46.394547 25522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:46.394763 25522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:59:46.399658 25522 server_base.cc:1034] running on GCE node
W20260808 22:59:46.399696 25531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:46.399724 25532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:46.399701 25534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:46.400035 25522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:46.400271 25522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:46.401413 25522 hybrid_clock.cc:648] HybridClock initialized: now 1786229986401395 us; error 35 us; skew 500 ppm
Aug 08 22:59:46 dist-test-slave-f0tn krb5kdc[25150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229986, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:46.403797 25522 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 22:59:46.404227 25522 webserver.cc:460] Webserver started at http://127.23.194.2:42731/ using document root <none> and password file <none>
I20260808 22:59:46.404408 25522 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:46.404448 25522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:46.404533 25522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:46.405588 25522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "e060e9abdafb4a7ea9e0103b746bb170"
format_stamp: "Formatted at 2026-08-08 22:59:46 on dist-test-slave-f0tn"
server_key: "868c95bf8272efedbc5af70a9dcc00ab"
server_key_iv: "49ea3107b097b980273237d0cbb29876"
server_key_version: "encryptionkey@0"
I20260808 22:59:46.405879 25522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "e060e9abdafb4a7ea9e0103b746bb170"
format_stamp: "Formatted at 2026-08-08 22:59:46 on dist-test-slave-f0tn"
server_key: "868c95bf8272efedbc5af70a9dcc00ab"
server_key_iv: "49ea3107b097b980273237d0cbb29876"
server_key_version: "encryptionkey@0"
I20260808 22:59:46.407032 25522 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:46.407680 25548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:46.407809 25522 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 22:59:46.407891 25522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "e060e9abdafb4a7ea9e0103b746bb170"
format_stamp: "Formatted at 2026-08-08 22:59:46 on dist-test-slave-f0tn"
server_key: "868c95bf8272efedbc5af70a9dcc00ab"
server_key_iv: "49ea3107b097b980273237d0cbb29876"
server_key_version: "encryptionkey@0"
I20260808 22:59:46.407948 25522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:46.426749 25522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:46.428894 25522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:46.429041 25522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:46.429280 25522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:59:46.429617 25522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:59:46.429653 25522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:46.429683 25522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:59:46.429708 25522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:46.435178 25522 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:44069
I20260808 22:59:46.435232 25733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:44069 every 8 connection(s)
I20260808 22:59:46.435523 25522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 22:59:46.436090 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 25522
I20260808 22:59:46.436194 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 22:59:46.436371 24328 external_mini_cluster.cc:1428] Setting key aca6bf95a858c5c79670dd20b7e62a81
Aug 08 22:59:46 dist-test-slave-f0tn krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229986, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.194.3@KRBTEST.COM; defaulting to no policy
I20260808 22:59:46.445102 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.436227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:49163 (local address 127.23.194.62:35143)
0808 22:59:46.436293 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:46.436295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:46.436704 (+ 409us) server_negotiation.cc:407] Connection header received
0808 22:59:46.437764 (+ 1060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:46.437765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:46.437776 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:46.437827 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:46.438436 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.439167 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.440151 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.440424 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.440490 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.442595 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:46.442596 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:46.442596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:46.442603 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:46.443803 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:46.444241 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:46.444242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:46.444242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:46.444288 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:46.444520 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:46.444521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:46.444521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:46.444613 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:46.444673 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:46.444970 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 22:59:46.445032 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 22:59:46.445271 25742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.436155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35143 (local address 127.23.194.2:49163)
0808 22:59:46.436579 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:46.436627 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:46.437655 (+ 1028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:46.437909 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:46.437913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:46.438161 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:46.438343 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:46.438347 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.439260 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.439261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:46.440050 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:46.440053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.440503 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.440504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:46.440608 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.441038 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:46.441046 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:46.442506 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:46.443883 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:46.443890 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:46.443892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:46.444174 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:46.444343 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:46.444343 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:46.444344 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:46.444470 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:46.444684 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:46.444690 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:46.444846 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 22:59:46.444957 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 22:59:46.445079 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":76,"threads_started":1}
I20260808 22:59:46.445751 25736 heartbeater.cc:347] Connected to a master server at 127.23.194.62:35143
I20260808 22:59:46.445832 25736 heartbeater.cc:464] Registering TS with master...
I20260808 22:59:46.445984 25736 heartbeater.cc:511] Master 127.23.194.62:35143 requested a full tablet report, sending...
I20260808 22:59:46.446300 25214 ts_manager.cc:194] Registered new tserver with Master: e060e9abdafb4a7ea9e0103b746bb170 (127.23.194.2:44069)
I20260808 22:59:46.446676 25214 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:49163
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
I20260808 22:59:46.460407 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35143
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:34301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:46.542224 25747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:46.542428 25747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:46.542452 25747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:46.543576 25747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:46.543617 25747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:46.543644 25747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 22:59:46.545145 25747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:34301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35143
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:46.545351 25747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:46.545575 25747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:46.550341 25758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:46.550367 25761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:46.550374 25759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:46.550458 25747 server_base.cc:1034] running on GCE node
I20260808 22:59:46.550696 25747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:46.550951 25747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:46.552088 25747 hybrid_clock.cc:648] HybridClock initialized: now 1786229986552073 us; error 28 us; skew 500 ppm
Aug 08 22:59:46 dist-test-slave-f0tn krb5kdc[25150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229986, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:46.554441 25747 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 22:59:46.554876 25747 webserver.cc:460] Webserver started at http://127.23.194.3:42463/ using document root <none> and password file <none>
I20260808 22:59:46.555073 25747 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:46.555120 25747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:46.555234 25747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:46.556226 25747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "1689a9ffc9fa4e798f0c64bd96280dcd"
format_stamp: "Formatted at 2026-08-08 22:59:46 on dist-test-slave-f0tn"
server_key: "0812e8cfdd0a36f58a99e640727c1309"
server_key_iv: "6fc4a90b24900387f002b1f10ae82879"
server_key_version: "encryptionkey@0"
I20260808 22:59:46.556555 25747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "1689a9ffc9fa4e798f0c64bd96280dcd"
format_stamp: "Formatted at 2026-08-08 22:59:46 on dist-test-slave-f0tn"
server_key: "0812e8cfdd0a36f58a99e640727c1309"
server_key_iv: "6fc4a90b24900387f002b1f10ae82879"
server_key_version: "encryptionkey@0"
I20260808 22:59:46.557883 25747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:59:46.558526 25770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:46.558679 25747 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 22:59:46.558763 25747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "1689a9ffc9fa4e798f0c64bd96280dcd"
format_stamp: "Formatted at 2026-08-08 22:59:46 on dist-test-slave-f0tn"
server_key: "0812e8cfdd0a36f58a99e640727c1309"
server_key_iv: "6fc4a90b24900387f002b1f10ae82879"
server_key_version: "encryptionkey@0"
I20260808 22:59:46.558820 25747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:46.567010 25747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:46.568608 25747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:46.568737 25747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:46.568928 25747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:59:46.569198 25747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:59:46.569231 25747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:46.569258 25747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:59:46.569283 25747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:46.581058 25747 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:37835
I20260808 22:59:46.581154 25943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:37835 every 8 connection(s)
I20260808 22:59:46.581473 25747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 22:59:46.584968 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 25747
I20260808 22:59:46.585088 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 22:59:46.585258 24328 external_mini_cluster.cc:1428] Setting key 2238c2e5f7201cdfa0b3cc6a58563923
Aug 08 22:59:46 dist-test-slave-f0tn krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229986, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:46.592183 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.582387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:58973 (local address 127.23.194.62:35143)
0808 22:59:46.582448 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:46.582451 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:46.582679 (+ 228us) server_negotiation.cc:407] Connection header received
0808 22:59:46.583851 (+ 1172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:46.583852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:46.583868 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:46.583922 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:46.584699 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.585442 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.586757 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.586965 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.587030 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.589465 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:46.589467 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:46.589467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:46.589484 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:46.590848 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:46.591254 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:46.591255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:46.591256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:46.591315 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:46.591552 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:46.591553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:46.591553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:46.591670 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:46.591733 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:46.591998 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 22:59:46.592086 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 22:59:46.592392 25946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.582352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35143 (local address 127.23.194.3:58973)
0808 22:59:46.582542 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:46.582579 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:46.583733 (+ 1154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:46.584013 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:46.584017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:46.584315 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:46.584596 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:46.584602 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.585555 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.585556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:46.586665 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:46.586668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.587043 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.587044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:46.587203 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.587794 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:46.587802 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:46.589352 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:46.590938 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:46.590944 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:46.590947 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:46.591180 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:46.591374 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:46.591375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:46.591375 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:46.591490 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:46.591730 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:46.591737 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:46.591873 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 22:59:46.591967 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 22:59:46.592135 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":69,"threads_started":1}
I20260808 22:59:46.593030 25944 heartbeater.cc:347] Connected to a master server at 127.23.194.62:35143
I20260808 22:59:46.593127 25944 heartbeater.cc:464] Registering TS with master...
I20260808 22:59:46.593302 25944 heartbeater.cc:511] Master 127.23.194.62:35143 requested a full tablet report, sending...
I20260808 22:59:46.593757 25214 ts_manager.cc:194] Registered new tserver with Master: 1689a9ffc9fa4e798f0c64bd96280dcd (127.23.194.3:37835)
I20260808 22:59:46.594120 25214 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:58973
I20260808 22:59:46.596860 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:59:46.604521 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.598816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54292 (local address 127.23.194.62:35143)
0808 22:59:46.598897 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:46.598899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:46.599132 (+ 233us) server_negotiation.cc:407] Connection header received
0808 22:59:46.599172 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:46.599173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:46.599182 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:46.599239 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:46.599841 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.600310 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.600975 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.601165 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.601222 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.602364 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:46.602365 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:46.602365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:46.602372 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:46.603659 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:46.603909 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:46.603910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:46.603910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:46.603958 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:46.604085 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:46.604085 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:46.604086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:46.604214 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:46.604284 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:46.604362 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 22:59:46.604422 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:59:46.607168 25214 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54292:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:59:46.608908 25214 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:59:46.615912 25959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.611892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59054 (local address 127.23.194.1:34039)
0808 22:59:46.612148 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:46.612152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:46.612163 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:59:46.612206 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:46.612208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:46.612223 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:46.612302 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:46.612433 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.613402 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.614814 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.615438 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.615511 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.615568 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:46.615780 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 22:59:46.615824 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":129,"threads_started":1}
I20260808 22:59:46.615932 25956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.611732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:34039 (local address 127.0.0.1:59054)
0808 22:59:46.611916 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:46.611952 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:46.612056 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:46.612305 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:46.612308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:46.612309 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:46.612360 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:46.612361 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.613594 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.613595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:46.614706 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:46.614710 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.615519 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.615520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:46.615661 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.615677 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 22:59:46.615748 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 22:59:46.615805 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":76,"threads_started":1}
I20260808 22:59:46.616232 25962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.611768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:37835 (local address 127.0.0.1:58974)
0808 22:59:46.612173 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:46.612182 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:46.612223 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:46.612535 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:46.612537 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:46.612539 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:46.612587 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:46.612588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.614043 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.614045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:46.615107 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:46.615109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.615996 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.615997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:46.616059 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.616063 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:59:46.616115 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 22:59:46.616163 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":52,"threads_started":1}
I20260808 22:59:46.616288 25960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.611961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58974 (local address 127.23.194.3:37835)
0808 22:59:46.612413 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:46.612416 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:46.612423 (+ 7us) server_negotiation.cc:407] Connection header received
0808 22:59:46.612449 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:46.612450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:46.612461 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:46.612516 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:46.612684 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.613865 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.615193 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.615916 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.615979 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.616035 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:46.616128 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 22:59:46.616165 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":75,"threads_started":1}
I20260808 22:59:46.616976 25408 tablet_service.cc:1467] Processing CreateTablet for tablet 0d63cd969e6743cb87e963bf1c95aee8 (DEFAULT_TABLE table=test-table [id=339c1682766c4da8afd6a52e6fba6c43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:46.617408 25408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d63cd969e6743cb87e963bf1c95aee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:46.617466 25957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.611732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:44069 (local address 127.0.0.1:55674)
0808 22:59:46.612134 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:46.612145 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:46.612208 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:46.613143 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:46.613144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:46.613147 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:46.613190 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:46.613191 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.614466 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.614467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:46.615425 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:46.615428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.617137 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.617138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:46.617277 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.617283 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:59:46.617340 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 22:59:46.617398 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":2,"thread_start_us":406,"threads_started":1}
I20260808 22:59:46.617628 25961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.611940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55674 (local address 127.23.194.2:44069)
0808 22:59:46.612692 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:46.612696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:46.612705 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:59:46.612890 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:46.612893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:46.612910 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:46.612984 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:46.613263 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.614308 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.616346 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.617039 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.617117 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.617174 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:46.617447 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 22:59:46.617500 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":110,"threads_started":1}
I20260808 22:59:46.618266 25630 tablet_service.cc:1467] Processing CreateTablet for tablet 0d63cd969e6743cb87e963bf1c95aee8 (DEFAULT_TABLE table=test-table [id=339c1682766c4da8afd6a52e6fba6c43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:46.618554 25630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d63cd969e6743cb87e963bf1c95aee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:46.621340 25963 tablet_bootstrap.cc:492] T 0d63cd969e6743cb87e963bf1c95aee8 P bade055e33b443b7ac55d830efacb509: Bootstrap starting.
I20260808 22:59:46.621838 25841 tablet_service.cc:1467] Processing CreateTablet for tablet 0d63cd969e6743cb87e963bf1c95aee8 (DEFAULT_TABLE table=test-table [id=339c1682766c4da8afd6a52e6fba6c43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:46.622210 25841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d63cd969e6743cb87e963bf1c95aee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:46.622594 25965 tablet_bootstrap.cc:492] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170: Bootstrap starting.
I20260808 22:59:46.622848 25963 tablet_bootstrap.cc:654] T 0d63cd969e6743cb87e963bf1c95aee8 P bade055e33b443b7ac55d830efacb509: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:46.623620 25965 tablet_bootstrap.cc:654] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:46.624066 25963 log.cc:824] T 0d63cd969e6743cb87e963bf1c95aee8 P bade055e33b443b7ac55d830efacb509: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:46.624552 25965 log.cc:824] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:46.625002 25963 tablet_bootstrap.cc:492] T 0d63cd969e6743cb87e963bf1c95aee8 P bade055e33b443b7ac55d830efacb509: No bootstrap required, opened a new log
I20260808 22:59:46.625077 25963 ts_tablet_manager.cc:1397] T 0d63cd969e6743cb87e963bf1c95aee8 P bade055e33b443b7ac55d830efacb509: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260808 22:59:46.625413 25965 tablet_bootstrap.cc:492] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170: No bootstrap required, opened a new log
I20260808 22:59:46.625495 25965 ts_tablet_manager.cc:1397] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:59:46.626621 25970 tablet_bootstrap.cc:492] T 0d63cd969e6743cb87e963bf1c95aee8 P 1689a9ffc9fa4e798f0c64bd96280dcd: Bootstrap starting.
I20260808 22:59:46.627779 25970 tablet_bootstrap.cc:654] T 0d63cd969e6743cb87e963bf1c95aee8 P 1689a9ffc9fa4e798f0c64bd96280dcd: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:46.627899 25963 raft_consensus.cc:348] T 0d63cd969e6743cb87e963bf1c95aee8 P bade055e33b443b7ac55d830efacb509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e060e9abdafb4a7ea9e0103b746bb170" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44069 } } peers { permanent_uuid: "bade055e33b443b7ac55d830efacb509" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34039 } } peers { permanent_uuid: "1689a9ffc9fa4e798f0c64bd96280dcd" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37835 } }
I20260808 22:59:46.628156 25963 raft_consensus.cc:374] T 0d63cd969e6743cb87e963bf1c95aee8 P bade055e33b443b7ac55d830efacb509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:46.628226 25963 raft_consensus.cc:729] T 0d63cd969e6743cb87e963bf1c95aee8 P bade055e33b443b7ac55d830efacb509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bade055e33b443b7ac55d830efacb509, State: Initialized, Role: FOLLOWER
I20260808 22:59:46.628351 25963 consensus_queue.cc:260] T 0d63cd969e6743cb87e963bf1c95aee8 P bade055e33b443b7ac55d830efacb509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e060e9abdafb4a7ea9e0103b746bb170" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44069 } } peers { permanent_uuid: "bade055e33b443b7ac55d830efacb509" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34039 } } peers { permanent_uuid: "1689a9ffc9fa4e798f0c64bd96280dcd" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37835 } }
I20260808 22:59:46.628400 25965 raft_consensus.cc:348] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e060e9abdafb4a7ea9e0103b746bb170" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44069 } } peers { permanent_uuid: "bade055e33b443b7ac55d830efacb509" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34039 } } peers { permanent_uuid: "1689a9ffc9fa4e798f0c64bd96280dcd" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37835 } }
I20260808 22:59:46.628710 25514 heartbeater.cc:503] Master 127.23.194.62:35143 was elected leader, sending a full tablet report...
I20260808 22:59:46.628719 25965 raft_consensus.cc:374] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:46.628760 25963 ts_tablet_manager.cc:1428] T 0d63cd969e6743cb87e963bf1c95aee8 P bade055e33b443b7ac55d830efacb509: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 22:59:46.628774 25965 raft_consensus.cc:729] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e060e9abdafb4a7ea9e0103b746bb170, State: Initialized, Role: FOLLOWER
I20260808 22:59:46.628798 25970 log.cc:824] T 0d63cd969e6743cb87e963bf1c95aee8 P 1689a9ffc9fa4e798f0c64bd96280dcd: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:46.628886 25965 consensus_queue.cc:260] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e060e9abdafb4a7ea9e0103b746bb170" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44069 } } peers { permanent_uuid: "bade055e33b443b7ac55d830efacb509" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34039 } } peers { permanent_uuid: "1689a9ffc9fa4e798f0c64bd96280dcd" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37835 } }
I20260808 22:59:46.629309 25736 heartbeater.cc:503] Master 127.23.194.62:35143 was elected leader, sending a full tablet report...
I20260808 22:59:46.629328 25965 ts_tablet_manager.cc:1428] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 22:59:46.630093 25970 tablet_bootstrap.cc:492] T 0d63cd969e6743cb87e963bf1c95aee8 P 1689a9ffc9fa4e798f0c64bd96280dcd: No bootstrap required, opened a new log
I20260808 22:59:46.630209 25970 ts_tablet_manager.cc:1397] T 0d63cd969e6743cb87e963bf1c95aee8 P 1689a9ffc9fa4e798f0c64bd96280dcd: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 22:59:46.632921 25970 raft_consensus.cc:348] T 0d63cd969e6743cb87e963bf1c95aee8 P 1689a9ffc9fa4e798f0c64bd96280dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e060e9abdafb4a7ea9e0103b746bb170" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44069 } } peers { permanent_uuid: "bade055e33b443b7ac55d830efacb509" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34039 } } peers { permanent_uuid: "1689a9ffc9fa4e798f0c64bd96280dcd" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37835 } }
I20260808 22:59:46.633172 25970 raft_consensus.cc:374] T 0d63cd969e6743cb87e963bf1c95aee8 P 1689a9ffc9fa4e798f0c64bd96280dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:46.633212 25970 raft_consensus.cc:729] T 0d63cd969e6743cb87e963bf1c95aee8 P 1689a9ffc9fa4e798f0c64bd96280dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1689a9ffc9fa4e798f0c64bd96280dcd, State: Initialized, Role: FOLLOWER
I20260808 22:59:46.633296 25970 consensus_queue.cc:260] T 0d63cd969e6743cb87e963bf1c95aee8 P 1689a9ffc9fa4e798f0c64bd96280dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e060e9abdafb4a7ea9e0103b746bb170" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44069 } } peers { permanent_uuid: "bade055e33b443b7ac55d830efacb509" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34039 } } peers { permanent_uuid: "1689a9ffc9fa4e798f0c64bd96280dcd" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37835 } }
I20260808 22:59:46.633605 25944 heartbeater.cc:503] Master 127.23.194.62:35143 was elected leader, sending a full tablet report...
I20260808 22:59:46.633643 25970 ts_tablet_manager.cc:1428] T 0d63cd969e6743cb87e963bf1c95aee8 P 1689a9ffc9fa4e798f0c64bd96280dcd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 22:59:46.640424 25977 raft_consensus.cc:482] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:59:46.640494 25977 raft_consensus.cc:504] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e060e9abdafb4a7ea9e0103b746bb170" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44069 } } peers { permanent_uuid: "bade055e33b443b7ac55d830efacb509" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34039 } } peers { permanent_uuid: "1689a9ffc9fa4e798f0c64bd96280dcd" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37835 } }
I20260808 22:59:46.640977 25977 leader_election.cc:288] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bade055e33b443b7ac55d830efacb509 (127.23.194.1:34039), 1689a9ffc9fa4e798f0c64bd96280dcd (127.23.194.3:37835)
I20260808 22:59:46.644217 25742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.641110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:34039 (local address 127.23.194.2:52429)
0808 22:59:46.641155 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:46.641166 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:46.641200 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:46.641586 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:46.641587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:46.641589 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:46.641740 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:46.641742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.642398 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.642399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:46.643231 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:46.643234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.643932 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.643933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:46.644072 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.644076 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:59:46.644126 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 22:59:46.644167 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 22:59:46.644272 25959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.641406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:52429 (local address 127.23.194.1:34039)
0808 22:59:46.641455 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:46.641457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:46.641467 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:59:46.641490 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:46.641491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:46.641503 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:46.641555 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:46.641808 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.642319 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.643315 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.643863 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.643922 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.643929 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:46.644166 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 22:59:46.644209 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 22:59:46.644474 25960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.641260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:56207 (local address 127.23.194.3:37835)
0808 22:59:46.641305 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:46.641307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:46.641460 (+ 153us) server_negotiation.cc:407] Connection header received
0808 22:59:46.641559 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:46.641560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:46.641571 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:46.641620 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:46.641879 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.642314 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.643623 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.644045 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.644119 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.644126 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:46.644374 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 22:59:46.644415 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 22:59:46.644587 25437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d63cd969e6743cb87e963bf1c95aee8" candidate_uuid: "e060e9abdafb4a7ea9e0103b746bb170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bade055e33b443b7ac55d830efacb509" is_pre_election: true
I20260808 22:59:46.644697 25979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.641136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:37835 (local address 127.23.194.2:56207)
0808 22:59:46.641365 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:46.641407 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:46.641490 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:46.641626 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:46.641627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:46.641628 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:46.641805 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:46.641807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.642391 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.642392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:46.643530 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:46.643534 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.644152 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.644154 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:46.644272 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.644277 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:59:46.644549 (+ 272us) client_negotiation.cc:240] Negotiation successful
0808 22:59:46.644592 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":131,"threads_started":1}
I20260808 22:59:46.644739 25437 raft_consensus.cc:2454] T 0d63cd969e6743cb87e963bf1c95aee8 P bade055e33b443b7ac55d830efacb509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e060e9abdafb4a7ea9e0103b746bb170 in term 0.
I20260808 22:59:46.645023 25554 leader_election.cc:302] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bade055e33b443b7ac55d830efacb509, e060e9abdafb4a7ea9e0103b746bb170; no voters:
I20260808 22:59:46.645046 25878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d63cd969e6743cb87e963bf1c95aee8" candidate_uuid: "e060e9abdafb4a7ea9e0103b746bb170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1689a9ffc9fa4e798f0c64bd96280dcd" is_pre_election: true
I20260808 22:59:46.645155 25977 raft_consensus.cc:2781] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:59:46.645218 25977 raft_consensus.cc:482] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:59:46.645218 25878 raft_consensus.cc:2454] T 0d63cd969e6743cb87e963bf1c95aee8 P 1689a9ffc9fa4e798f0c64bd96280dcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e060e9abdafb4a7ea9e0103b746bb170 in term 0.
I20260808 22:59:46.645248 25977 raft_consensus.cc:3037] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:46.646261 25977 raft_consensus.cc:504] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e060e9abdafb4a7ea9e0103b746bb170" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44069 } } peers { permanent_uuid: "bade055e33b443b7ac55d830efacb509" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34039 } } peers { permanent_uuid: "1689a9ffc9fa4e798f0c64bd96280dcd" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37835 } }
I20260808 22:59:46.646368 25977 leader_election.cc:288] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170 [CANDIDATE]: Term 1 election: Requested vote from peers bade055e33b443b7ac55d830efacb509 (127.23.194.1:34039), 1689a9ffc9fa4e798f0c64bd96280dcd (127.23.194.3:37835)
I20260808 22:59:46.646497 25878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d63cd969e6743cb87e963bf1c95aee8" candidate_uuid: "e060e9abdafb4a7ea9e0103b746bb170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1689a9ffc9fa4e798f0c64bd96280dcd"
I20260808 22:59:46.646567 25878 raft_consensus.cc:3037] T 0d63cd969e6743cb87e963bf1c95aee8 P 1689a9ffc9fa4e798f0c64bd96280dcd [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:46.646560 25437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d63cd969e6743cb87e963bf1c95aee8" candidate_uuid: "e060e9abdafb4a7ea9e0103b746bb170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bade055e33b443b7ac55d830efacb509"
I20260808 22:59:46.646629 25437 raft_consensus.cc:3037] T 0d63cd969e6743cb87e963bf1c95aee8 P bade055e33b443b7ac55d830efacb509 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:46.647564 25437 raft_consensus.cc:2454] T 0d63cd969e6743cb87e963bf1c95aee8 P bade055e33b443b7ac55d830efacb509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e060e9abdafb4a7ea9e0103b746bb170 in term 1.
I20260808 22:59:46.647727 25554 leader_election.cc:302] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bade055e33b443b7ac55d830efacb509, e060e9abdafb4a7ea9e0103b746bb170; no voters:
I20260808 22:59:46.647776 25878 raft_consensus.cc:2454] T 0d63cd969e6743cb87e963bf1c95aee8 P 1689a9ffc9fa4e798f0c64bd96280dcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e060e9abdafb4a7ea9e0103b746bb170 in term 1.
I20260808 22:59:46.647801 25977 raft_consensus.cc:2781] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:46.647938 25977 raft_consensus.cc:686] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170 [term 1 LEADER]: Becoming Leader. State: Replica: e060e9abdafb4a7ea9e0103b746bb170, State: Running, Role: LEADER
I20260808 22:59:46.648048 25977 consensus_queue.cc:237] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e060e9abdafb4a7ea9e0103b746bb170" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44069 } } peers { permanent_uuid: "bade055e33b443b7ac55d830efacb509" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34039 } } peers { permanent_uuid: "1689a9ffc9fa4e798f0c64bd96280dcd" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37835 } }
I20260808 22:59:46.648859 25214 catalog_manager.cc:5495] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170 reported cstate change: term changed from 0 to 1, leader changed from <none> to e060e9abdafb4a7ea9e0103b746bb170 (127.23.194.2). New cstate: current_term: 1 leader_uuid: "e060e9abdafb4a7ea9e0103b746bb170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e060e9abdafb4a7ea9e0103b746bb170" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bade055e33b443b7ac55d830efacb509" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1689a9ffc9fa4e798f0c64bd96280dcd" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37835 } health_report { overall_health: UNKNOWN } } }
Aug 08 22:59:46 dist-test-slave-f0tn krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.1@KRBTEST.COM
I20260808 22:59:46.665273 25959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.657605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59070 (local address 127.23.194.1:34039)
0808 22:59:46.657652 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:46.657666 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:46.657763 (+ 97us) server_negotiation.cc:407] Connection header received
0808 22:59:46.657827 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:46.657828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:46.657837 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:46.657884 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:46.658433 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.658888 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.659565 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.659760 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.659814 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.662276 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:46.662283 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:46.662284 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:46.662291 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:46.664188 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:46.664508 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:46.664511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:46.664511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:46.664566 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:46.664707 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:46.664707 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:46.664707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:46.664840 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:46.664939 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:46.665115 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 22:59:46.665182 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
Aug 08 22:59:46 dist-test-slave-f0tn krb5kdc[25150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229986, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260808 22:59:46.686034 25739 tablet.cc:2369] T 0d63cd969e6743cb87e963bf1c95aee8 P e060e9abdafb4a7ea9e0103b746bb170: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 22:59:46 dist-test-slave-f0tn krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229986, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.194.1@KRBTEST.COM
I20260808 22:59:46.688939 25959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.682894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59072 (local address 127.23.194.1:34039)
0808 22:59:46.682959 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:46.682961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:46.682987 (+ 26us) server_negotiation.cc:407] Connection header received
0808 22:59:46.683053 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:46.683053 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:46.683062 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:46.683107 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:46.683603 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.684053 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.684698 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.684867 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.684911 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.687067 (+ 2156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:46.687069 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:46.687069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:46.687075 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:46.688139 (+ 1064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:46.688410 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:46.688410 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:46.688411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:46.688453 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:46.688571 (+ 118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:46.688571 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:46.688571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:46.688647 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:46.688708 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:46.688825 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 22:59:46.688877 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 22:59:46 dist-test-slave-f0tn krb5kdc[25150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 22:59:46 dist-test-slave-f0tn krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.1@KRBTEST.COM
I20260808 22:59:46.709779 25959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:46.703825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59082 (local address 127.23.194.1:34039)
0808 22:59:46.703877 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:46.703879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:46.703964 (+ 85us) server_negotiation.cc:407] Connection header received
0808 22:59:46.704006 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:46.704007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:46.704015 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:46.704059 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:46.704497 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.704941 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.705676 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:46.705855 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:46.705902 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:46.707788 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:46.707789 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:46.707789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:46.707796 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:46.708965 (+ 1169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:46.709209 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:46.709210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:46.709210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:46.709254 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:46.709371 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:46.709372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:46.709372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:46.709479 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:46.709546 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:46.709642 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 22:59:46.709695 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260808 22:59:46.710522 25378 tablet_service.cc:2908] Rejecting scan request for tablet 0d63cd969e6743cb87e963bf1c95aee8: Uninitialized: clean time has not yet been initialized
I20260808 22:59:46.711453 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 25309
I20260808 22:59:46.730075 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 25522
I20260808 22:59:46.743103 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 25747
I20260808 22:59:46.749706 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 25168
2026-08-08T22:59:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (3366 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:46 dist-test-slave-f0tn krb5kdc[25998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:46 dist-test-slave-f0tn krb5kdc[25998](info): set up 2 sockets
Aug 08 22:59:46 dist-test-slave-f0tn krb5kdc[25998](info): commencing operation
krb5kdc: starting...
W20260808 22:59:47.890053 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 22:59:47 dist-test-slave-f0tn krb5kdc[25998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 22:59:48.004462 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:36133
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:44489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:36133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:48.085245 26018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:48.085472 26018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:48.085505 26018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:48.086611 26018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:48.086650 26018 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 22:59:48.086668 26018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:48.086684 26018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:48.086701 26018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:48.086723 26018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:48.088147 26018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:44489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:36133
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:36133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:48.088376 26018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:48.088572 26018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:48.093261 26028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:48.093254 26031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:48.093307 26018 server_base.cc:1034] running on GCE node
W20260808 22:59:48.093246 26027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:48.093714 26018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:48.093977 26018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:48.095153 26018 hybrid_clock.cc:648] HybridClock initialized: now 1786229988095134 us; error 32 us; skew 500 ppm
Aug 08 22:59:48 dist-test-slave-f0tn krb5kdc[25998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229988, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:48.097528 26018 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 22:59:48.097906 26018 webserver.cc:460] Webserver started at http://127.23.194.62:39017/ using document root <none> and password file <none>
I20260808 22:59:48.098078 26018 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:48.098115 26018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:48.098204 26018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:48.099211 26018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "c5b289853643455c896139aab7673364"
format_stamp: "Formatted at 2026-08-08 22:59:48 on dist-test-slave-f0tn"
server_key: "93828a374f0d88f12c4863aa839806c6"
server_key_iv: "41fb85851d23cda6556d23321abafa1b"
server_key_version: "encryptionkey@0"
I20260808 22:59:48.099499 26018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "c5b289853643455c896139aab7673364"
format_stamp: "Formatted at 2026-08-08 22:59:48 on dist-test-slave-f0tn"
server_key: "93828a374f0d88f12c4863aa839806c6"
server_key_iv: "41fb85851d23cda6556d23321abafa1b"
server_key_version: "encryptionkey@0"
I20260808 22:59:48.100668 26018 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 22:59:48.101280 26042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:48.101455 26018 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:48.101542 26018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "c5b289853643455c896139aab7673364"
format_stamp: "Formatted at 2026-08-08 22:59:48 on dist-test-slave-f0tn"
server_key: "93828a374f0d88f12c4863aa839806c6"
server_key_iv: "41fb85851d23cda6556d23321abafa1b"
server_key_version: "encryptionkey@0"
I20260808 22:59:48.101596 26018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:48.118038 26018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:48.120079 26018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:48.120194 26018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:48.124740 26018 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:36133
I20260808 22:59:48.124812 26140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:36133 every 8 connection(s)
I20260808 22:59:48.125092 26018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:48.125761 26141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:48.127919 26141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364: Bootstrap starting.
I20260808 22:59:48.128470 26141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:48.128731 26141 log.cc:824] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:48.129319 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 26018
I20260808 22:59:48.129434 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 22:59:48.129444 26141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364: No bootstrap required, opened a new log
I20260808 22:59:48.129607 24328 external_mini_cluster.cc:1428] Setting key b9a8a01d6527a2db06624980a9b22cec
I20260808 22:59:48.131819 26141 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5b289853643455c896139aab7673364" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36133 } }
I20260808 22:59:48.131955 26141 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:48.131989 26141 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5b289853643455c896139aab7673364, State: Initialized, Role: FOLLOWER
I20260808 22:59:48.132130 26141 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5b289853643455c896139aab7673364" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36133 } }
I20260808 22:59:48.132206 26141 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:48.132238 26141 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:48.132287 26141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:48.132779 26141 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5b289853643455c896139aab7673364" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36133 } }
I20260808 22:59:48.132877 26141 leader_election.cc:302] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5b289853643455c896139aab7673364; no voters:
I20260808 22:59:48.133044 26141 leader_election.cc:288] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:48.133082 26153 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:48.133194 26153 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364 [term 1 LEADER]: Becoming Leader. State: Replica: c5b289853643455c896139aab7673364, State: Running, Role: LEADER
I20260808 22:59:48.133298 26141 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:48.133333 26153 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5b289853643455c896139aab7673364" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36133 } }
I20260808 22:59:48.133769 26158 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5b289853643455c896139aab7673364" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5b289853643455c896139aab7673364" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36133 } } }
I20260808 22:59:48.133850 26158 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:48.133996 26159 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5b289853643455c896139aab7673364. Latest consensus state: current_term: 1 leader_uuid: "c5b289853643455c896139aab7673364" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5b289853643455c896139aab7673364" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36133 } } }
I20260808 22:59:48.134052 26159 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:48.134510 26166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:48.135032 26166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 22:59:48 dist-test-slave-f0tn krb5kdc[25998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:48.137352 26166 catalog_manager.cc:1347] Generated new cluster ID: be56f49a96d34ecdbab6836d395e8e2d
I20260808 22:59:48.137421 26166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:59:48.141428 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.130094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47264 (local address 127.23.194.62:36133)
0808 22:59:48.130274 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:48.130282 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:48.130307 (+ 25us) server_negotiation.cc:407] Connection header received
0808 22:59:48.131786 (+ 1479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:48.131792 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:48.131833 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:48.131950 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:48.132600 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.133765 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.134789 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.135338 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.135444 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.137813 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:48.137817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:48.137817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:48.137829 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:48.139720 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:48.140079 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:48.140082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:48.140083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:48.140218 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:48.140388 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:48.140389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:48.140389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:48.140704 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:48.140851 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:48.140950 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 22:59:48.141201 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":85,"threads_started":1}
I20260808 22:59:48.151664 26166 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:48.152356 26166 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:48.159368 26166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364: Generated new TSK 0
I20260808 22:59:48.159539 26166 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
I20260808 22:59:48.185046 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:36133
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:44489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:48.264484 26187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:48.264684 26187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:48.264706 26187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:48.265866 26187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:48.265905 26187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:48.265924 26187 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:59:48.265941 26187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 22:59:48.267344 26187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:44489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.194.62:36133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:48.267568 26187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:48.267776 26187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:48.272503 26204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:48.272513 26202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:48.272523 26201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:48.272754 26187 server_base.cc:1034] running on GCE node
I20260808 22:59:48.272939 26187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:48.273147 26187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:48.274286 26187 hybrid_clock.cc:648] HybridClock initialized: now 1786229988274266 us; error 33 us; skew 500 ppm
Aug 08 22:59:48 dist-test-slave-f0tn krb5kdc[25998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229988, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:48.276693 26187 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 22:59:48.277082 26187 webserver.cc:460] Webserver started at http://127.23.194.1:40331/ using document root <none> and password file <none>
I20260808 22:59:48.277271 26187 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:48.277310 26187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:48.277432 26187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:48.278357 26187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "a558ea4728844e59a1f7b787b2b1a7ff"
format_stamp: "Formatted at 2026-08-08 22:59:48 on dist-test-slave-f0tn"
server_key: "9fd26520d50e4b61a3935f01fb3a539d"
server_key_iv: "28650cf28d7ddd94a24b2f790362f966"
server_key_version: "encryptionkey@0"
I20260808 22:59:48.278640 26187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "a558ea4728844e59a1f7b787b2b1a7ff"
format_stamp: "Formatted at 2026-08-08 22:59:48 on dist-test-slave-f0tn"
server_key: "9fd26520d50e4b61a3935f01fb3a539d"
server_key_iv: "28650cf28d7ddd94a24b2f790362f966"
server_key_version: "encryptionkey@0"
I20260808 22:59:48.279899 26187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:59:48.280493 26213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:48.280635 26187 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 22:59:48.280715 26187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "a558ea4728844e59a1f7b787b2b1a7ff"
format_stamp: "Formatted at 2026-08-08 22:59:48 on dist-test-slave-f0tn"
server_key: "9fd26520d50e4b61a3935f01fb3a539d"
server_key_iv: "28650cf28d7ddd94a24b2f790362f966"
server_key_version: "encryptionkey@0"
I20260808 22:59:48.280778 26187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:48.294167 26187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:48.296160 26187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:48.296283 26187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:48.296774 26187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:59:48.296818 26187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:48.296841 26187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:59:48.296854 26187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:59:48 dist-test-slave-f0tn krb5kdc[25998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229988, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:48.303715 26187 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:41423
I20260808 22:59:48.303786 26401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:41423 every 8 connection(s)
I20260808 22:59:48.304111 26187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 22:59:48.305909 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.297258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:37371 (local address 127.23.194.62:36133)
0808 22:59:48.297307 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:48.297309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:48.297549 (+ 240us) server_negotiation.cc:407] Connection header received
0808 22:59:48.298011 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:48.298013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:48.298025 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:48.298084 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:48.298958 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.299488 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.300938 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.301159 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.301219 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.303255 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:48.303256 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:48.303256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:48.303267 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:48.304784 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:48.305149 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:48.305149 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:48.305150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:48.305197 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:48.305365 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:48.305366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:48.305366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:48.305517 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:48.305568 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:48.305783 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 22:59:48.305846 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 22:59:48.306054 26225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.297252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:36133 (local address 127.23.194.1:37371)
0808 22:59:48.297435 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:48.297476 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:48.297918 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:48.298189 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:48.298194 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:48.298555 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:48.298841 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.298847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.299577 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.299578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.300834 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.300838 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.301249 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.301250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.301362 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.301840 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:48.301850 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:48.303165 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:48.304847 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:48.304850 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:48.304852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:48.305084 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:48.305238 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:48.305239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:48.305239 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:48.305327 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:48.305565 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:48.305569 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:48.305685 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 22:59:48.305764 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 22:59:48.305873 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":77,"threads_started":1}
I20260808 22:59:48.306512 26402 heartbeater.cc:347] Connected to a master server at 127.23.194.62:36133
I20260808 22:59:48.306604 26402 heartbeater.cc:464] Registering TS with master...
I20260808 22:59:48.306787 26402 heartbeater.cc:511] Master 127.23.194.62:36133 requested a full tablet report, sending...
I20260808 22:59:48.307307 26066 ts_manager.cc:194] Registered new tserver with Master: a558ea4728844e59a1f7b787b2b1a7ff (127.23.194.1:41423)
I20260808 22:59:48.307955 26066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:37371
I20260808 22:59:48.309296 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 26187
I20260808 22:59:48.309405 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 22:59:48.309547 24328 external_mini_cluster.cc:1428] Setting key b5f84f0aff24614b89b9752bd11079b7
I20260808 22:59:48.313081 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.307555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:57795 (local address 127.23.194.62:36133)
0808 22:59:48.308132 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:48.308134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:48.308141 (+ 7us) server_negotiation.cc:407] Connection header received
0808 22:59:48.308158 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:48.308159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:48.308168 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:48.308218 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:48.308544 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.309064 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.309818 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.310047 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.310115 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.310879 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:48.310881 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:48.310881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:48.310888 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:48.312134 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:48.312472 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:48.312472 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:48.312472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:48.312515 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:48.312673 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:48.312673 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:48.312673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:48.312754 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:48.312794 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:48.312927 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 22:59:48.312981 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558}
I20260808 22:59:48.313102 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.307492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:36133 (local address 127.23.194.1:57795)
0808 22:59:48.307602 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:48.307611 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:48.307660 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:48.308229 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:48.308230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:48.308405 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:48.308466 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.308468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.309136 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.309137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.309737 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.309739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.310114 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.310115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.310185 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.310547 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:48.310551 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:48.310803 (+ 252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:48.312208 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:48.312209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:48.312210 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:48.312408 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:48.312563 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:48.312563 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:48.312564 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:48.312625 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:48.312801 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:48.312801 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:48.312849 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 22:59:48.312915 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 22:59:48.313004 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for kudu/127.23.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
I20260808 22:59:48.332887 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:36133
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:44489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:48.413481 26412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:48.413686 26412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:48.413709 26412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:48.414836 26412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:48.414875 26412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:48.414896 26412 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:59:48.414925 26412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 22:59:48.416600 26412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:44489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.194.62:36133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:48.416836 26412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:48.417105 26412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:48.421965 26420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:48.421988 26412 server_base.cc:1034] running on GCE node
W20260808 22:59:48.421972 26423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:48.421989 26418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:48.422425 26412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:48.422658 26412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:48.423787 26412 hybrid_clock.cc:648] HybridClock initialized: now 1786229988423778 us; error 26 us; skew 500 ppm
Aug 08 22:59:48 dist-test-slave-f0tn krb5kdc[25998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229988, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:48.426081 26412 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 22:59:48.426479 26412 webserver.cc:460] Webserver started at http://127.23.194.2:34689/ using document root <none> and password file <none>
I20260808 22:59:48.426679 26412 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:48.426726 26412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:48.426846 26412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:48.427870 26412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "899db6005d564ab5aa986c4607a5abd9"
format_stamp: "Formatted at 2026-08-08 22:59:48 on dist-test-slave-f0tn"
server_key: "ac0180a5740d40173a1c73fce3c5af63"
server_key_iv: "e0854816723ddcefd8c04875014bb8ce"
server_key_version: "encryptionkey@0"
I20260808 22:59:48.428189 26412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "899db6005d564ab5aa986c4607a5abd9"
format_stamp: "Formatted at 2026-08-08 22:59:48 on dist-test-slave-f0tn"
server_key: "ac0180a5740d40173a1c73fce3c5af63"
server_key_iv: "e0854816723ddcefd8c04875014bb8ce"
server_key_version: "encryptionkey@0"
I20260808 22:59:48.429370 26412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:59:48.430076 26438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:48.430219 26412 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 22:59:48.430301 26412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "899db6005d564ab5aa986c4607a5abd9"
format_stamp: "Formatted at 2026-08-08 22:59:48 on dist-test-slave-f0tn"
server_key: "ac0180a5740d40173a1c73fce3c5af63"
server_key_iv: "e0854816723ddcefd8c04875014bb8ce"
server_key_version: "encryptionkey@0"
I20260808 22:59:48.430363 26412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:48.443787 26412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:48.445348 26412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:48.445497 26412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:48.445940 26412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:59:48.445982 26412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:48.446009 26412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:59:48.446025 26412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:48.452103 26412 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:45007
I20260808 22:59:48.452163 26609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:45007 every 8 connection(s)
Aug 08 22:59:48 dist-test-slave-f0tn krb5kdc[25998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229988, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:48.452467 26412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 22:59:48.455062 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.446378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:52989 (local address 127.23.194.62:36133)
0808 22:59:48.446425 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:48.446427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:48.446636 (+ 209us) server_negotiation.cc:407] Connection header received
0808 22:59:48.447049 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:48.447051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:48.447065 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:48.447133 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:48.447867 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.448609 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.450023 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.450314 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.450394 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.452782 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:48.452784 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:48.452784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:48.452792 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:48.454069 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:48.454417 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:48.454418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:48.454418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:48.454463 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:48.454630 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:48.454631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:48.454631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:48.454722 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:48.454765 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:48.454947 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 22:59:48.454999 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 22:59:48.455206 26456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.446368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:36133 (local address 127.23.194.2:52989)
0808 22:59:48.446512 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:48.446545 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:48.446964 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:48.447234 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:48.447238 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:48.447503 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:48.447760 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.447767 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.448695 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.448696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.449937 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.449940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.450406 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.450407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.450542 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.451003 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:48.451013 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:48.452678 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:48.454159 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:48.454163 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:48.454164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:48.454363 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:48.454508 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:48.454508 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:48.454508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:48.454590 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:48.454763 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:48.454765 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:48.454861 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 22:59:48.454932 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 22:59:48.455039 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":83,"threads_started":1}
I20260808 22:59:48.455662 26612 heartbeater.cc:347] Connected to a master server at 127.23.194.62:36133
I20260808 22:59:48.455742 26612 heartbeater.cc:464] Registering TS with master...
I20260808 22:59:48.455897 26612 heartbeater.cc:511] Master 127.23.194.62:36133 requested a full tablet report, sending...
I20260808 22:59:48.456234 26070 ts_manager.cc:194] Registered new tserver with Master: 899db6005d564ab5aa986c4607a5abd9 (127.23.194.2:45007)
I20260808 22:59:48.456701 26070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:52989
I20260808 22:59:48.457433 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 26412
I20260808 22:59:48.457515 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 22:59:48.457683 24328 external_mini_cluster.cc:1428] Setting key 862baa8f5e276a3d103659d6c9ef8549
I20260808 22:59:48.461659 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.456616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:53223 (local address 127.23.194.62:36133)
0808 22:59:48.456661 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:48.456663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:48.456728 (+ 65us) server_negotiation.cc:407] Connection header received
0808 22:59:48.456772 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:48.456773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:48.456782 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:48.456828 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:48.457136 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.457785 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.458527 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.458751 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.458818 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.459531 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:48.459532 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:48.459532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:48.459538 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:48.460632 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:48.460964 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:48.460965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:48.460966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:48.461019 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:48.461201 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:48.461201 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:48.461201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:48.461322 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:48.461367 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:48.461533 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 22:59:48.461590 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 22:59:48.461704 26622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.456560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:36133 (local address 127.23.194.2:53223)
0808 22:59:48.456676 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:48.456686 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:48.456733 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:48.456849 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:48.456850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:48.457022 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:48.457072 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.457073 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.457878 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.457880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.458461 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.458463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.458814 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.458815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.458866 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.459210 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:48.459215 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:48.459461 (+ 246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:48.460706 (+ 1245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:48.460706 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:48.460707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:48.460889 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:48.461078 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:48.461078 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:48.461079 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:48.461145 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:48.461373 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:48.461400 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:48.461445 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 22:59:48.461511 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 22:59:48.461603 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for kudu/127.23.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
I20260808 22:59:48.480610 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:36133
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:44489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:48.562673 26633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:48.562860 26633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:48.562881 26633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:48.564162 26633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:48.564229 26633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:48.564256 26633 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:59:48.564293 26633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 22:59:48.565834 26633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:44489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.194.62:36133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:48.566066 26633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:48.566278 26633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:48.571146 26642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:48.571183 26643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:48.571182 26633 server_base.cc:1034] running on GCE node
W20260808 22:59:48.571151 26646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:48.571468 26633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:48.571683 26633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:48.572817 26633 hybrid_clock.cc:648] HybridClock initialized: now 1786229988572804 us; error 27 us; skew 500 ppm
Aug 08 22:59:48 dist-test-slave-f0tn krb5kdc[25998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229988, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:48.575165 26633 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 22:59:48.575554 26633 webserver.cc:460] Webserver started at http://127.23.194.3:43803/ using document root <none> and password file <none>
I20260808 22:59:48.575766 26633 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:48.575812 26633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:48.575937 26633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:48.576900 26633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "ba03fcd319324788b94267712f05276d"
format_stamp: "Formatted at 2026-08-08 22:59:48 on dist-test-slave-f0tn"
server_key: "13dbe48e6916b6fdc3fffd43fc6599ca"
server_key_iv: "d90884e3fe1ec8700ddaea58bd3b0885"
server_key_version: "encryptionkey@0"
I20260808 22:59:48.577219 26633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "ba03fcd319324788b94267712f05276d"
format_stamp: "Formatted at 2026-08-08 22:59:48 on dist-test-slave-f0tn"
server_key: "13dbe48e6916b6fdc3fffd43fc6599ca"
server_key_iv: "d90884e3fe1ec8700ddaea58bd3b0885"
server_key_version: "encryptionkey@0"
I20260808 22:59:48.578431 26633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:48.579056 26658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:48.579216 26633 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 22:59:48.579298 26633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "ba03fcd319324788b94267712f05276d"
format_stamp: "Formatted at 2026-08-08 22:59:48 on dist-test-slave-f0tn"
server_key: "13dbe48e6916b6fdc3fffd43fc6599ca"
server_key_iv: "d90884e3fe1ec8700ddaea58bd3b0885"
server_key_version: "encryptionkey@0"
I20260808 22:59:48.579362 26633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:48.592343 26633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:48.594550 26633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:48.594694 26633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:48.595202 26633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:59:48.595238 26633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:48.595259 26633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:59:48.595271 26633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:59:48 dist-test-slave-f0tn krb5kdc[25998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229988, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:48.602100 26633 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:33829
I20260808 22:59:48.602191 26833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:33829 every 8 connection(s)
I20260808 22:59:48.602492 26633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 22:59:48.604593 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.595792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:47511 (local address 127.23.194.62:36133)
0808 22:59:48.595859 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:48.595861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:48.596011 (+ 150us) server_negotiation.cc:407] Connection header received
0808 22:59:48.596537 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:48.596538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:48.596552 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:48.596622 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:48.597481 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.598241 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.599256 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.599550 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.599625 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.601931 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:48.601933 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:48.601933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:48.601942 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:48.603277 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:48.603675 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:48.603676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:48.603677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:48.603737 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:48.603975 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:48.603976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:48.603976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:48.604096 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:48.604152 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:48.604418 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 22:59:48.604504 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:59:48.604753 26672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.595711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:36133 (local address 127.23.194.3:47511)
0808 22:59:48.595878 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:48.595919 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:48.596421 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:48.596729 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:48.596733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:48.597067 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:48.597348 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.597354 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.598329 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.598330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.599189 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.599192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.599649 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.599650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.599784 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.600237 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:48.600245 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:48.601834 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:48.603352 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:48.603359 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:48.603361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:48.603597 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:48.603791 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:48.603792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:48.603792 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:48.603917 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:48.604151 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:48.604165 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:48.604297 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 22:59:48.604386 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 22:59:48.604546 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260808 22:59:48.605168 26834 heartbeater.cc:347] Connected to a master server at 127.23.194.62:36133
I20260808 22:59:48.605185 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 26633
I20260808 22:59:48.605247 26834 heartbeater.cc:464] Registering TS with master...
I20260808 22:59:48.605264 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 22:59:48.605448 26834 heartbeater.cc:511] Master 127.23.194.62:36133 requested a full tablet report, sending...
I20260808 22:59:48.605473 24328 external_mini_cluster.cc:1428] Setting key 39f1cea4433c9cd7e9d5d769d64fb3e0
I20260808 22:59:48.605887 26070 ts_manager.cc:194] Registered new tserver with Master: ba03fcd319324788b94267712f05276d (127.23.194.3:33829)
I20260808 22:59:48.606312 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:59:48.606367 26070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:47511
I20260808 22:59:48.613437 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.606282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:44973 (local address 127.23.194.62:36133)
0808 22:59:48.606545 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:48.606547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:48.608460 (+ 1913us) server_negotiation.cc:407] Connection header received
0808 22:59:48.608514 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:48.608515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:48.608525 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:48.608576 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:48.608883 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.609343 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.610018 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.610203 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.610257 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.611115 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:48.611116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:48.611117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:48.611124 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:48.612294 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:48.612667 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:48.612668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:48.612669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:48.612726 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:48.612974 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:48.612975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:48.612975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:48.613070 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:48.613120 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:48.613280 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 22:59:48.613339 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241}
I20260808 22:59:48.613494 26847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.606255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:36133 (local address 127.23.194.3:44973)
0808 22:59:48.608390 (+ 2135us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:48.608401 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:48.608459 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:48.608581 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:48.608582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:48.608748 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:48.608818 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.608820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.609440 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.609441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.609951 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.609953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.610261 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.610261 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.610337 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.610758 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:48.610762 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:48.611039 (+ 277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:48.612380 (+ 1341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:48.612381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:48.612382 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:48.612584 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:48.612825 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:48.612826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:48.612826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:48.612918 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:48.613122 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:48.613123 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:48.613193 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 22:59:48.613263 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 22:59:48.613369 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2109,"thread_start_us":2070,"threads_started":1}
Aug 08 22:59:48 dist-test-slave-f0tn krb5kdc[25998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229988, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 22:59:48 dist-test-slave-f0tn krb5kdc[25998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229988, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:48.628661 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.622915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47272 (local address 127.23.194.62:36133)
0808 22:59:48.622965 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:48.622967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:48.622973 (+ 6us) server_negotiation.cc:407] Connection header received
0808 22:59:48.623021 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:48.623021 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:48.623029 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:48.623073 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:48.623536 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.623990 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.624622 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.624788 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.624828 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.626758 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:48.626759 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:48.626759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:48.626766 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:48.627918 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:48.628176 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:48.628177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:48.628177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:48.628219 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:48.628334 (+ 115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:48.628334 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:48.628335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:48.628417 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:48.628481 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:48.628545 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 22:59:48.628599 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 22:59:48.630081 26070 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47272:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:59:48.631522 26070 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:59:48.637638 26861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.633659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:41423 (local address 127.0.0.1:37312)
0808 22:59:48.633984 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:48.634022 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:48.634094 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:48.634441 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:48.634441 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:48.634444 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:48.634482 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.634483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.635755 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.635756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.636590 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.636593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.637321 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.637322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.637417 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.637434 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 22:59:48.637507 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 22:59:48.637559 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":1,"thread_start_us":64,"threads_started":1}
I20260808 22:59:48.637674 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.633847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37312 (local address 127.23.194.1:41423)
0808 22:59:48.634243 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:48.634246 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:48.634256 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:59:48.634291 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:48.634293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:48.634308 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:48.634413 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:48.634564 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.635655 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.636680 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.637243 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.637321 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.637448 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:48.637524 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 22:59:48.637563 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":93,"threads_started":1}
I20260808 22:59:48.638099 26867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.633959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53870 (local address 127.23.194.3:33829)
0808 22:59:48.634205 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:48.634209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:48.634612 (+ 403us) server_negotiation.cc:407] Connection header received
0808 22:59:48.634654 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:48.634656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:48.634672 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:48.634918 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:48.635045 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.635702 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.637041 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.637751 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.637817 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.637874 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:48.637976 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 22:59:48.638012 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":71,"threads_started":1}
I20260808 22:59:48.638384 26858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.633688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:45007 (local address 127.0.0.1:52062)
0808 22:59:48.634051 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:48.634067 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:48.634121 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:48.634359 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:48.634362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:48.634364 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:48.634429 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.634431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.635738 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.635740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.637193 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.637195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.638180 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.638181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.638242 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.638245 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:59:48.638298 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 22:59:48.638335 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":67,"thread_start_us":65,"threads_started":1}
I20260808 22:59:48.638461 26859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.633780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52062 (local address 127.23.194.2:45007)
0808 22:59:48.634186 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:48.634190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:48.634200 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:59:48.634236 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:48.634238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:48.634250 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:48.634332 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:48.634513 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.635663 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.637263 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.638034 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.638092 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.638145 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:48.638327 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 22:59:48.638370 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":91,"threads_started":1}
I20260808 22:59:48.638636 26286 tablet_service.cc:1467] Processing CreateTablet for tablet 7e6f3eb25c154f4dbae9f38cbfd32418 (DEFAULT_TABLE table=test-table [id=c1d8897c2e5546558ee1605463c5d56f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:48.639005 26286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e6f3eb25c154f4dbae9f38cbfd32418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:48.639034 26511 tablet_service.cc:1467] Processing CreateTablet for tablet 7e6f3eb25c154f4dbae9f38cbfd32418 (DEFAULT_TABLE table=test-table [id=c1d8897c2e5546558ee1605463c5d56f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:48.639371 26511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e6f3eb25c154f4dbae9f38cbfd32418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:48.642158 26857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.633884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:33829 (local address 127.0.0.1:53870)
0808 22:59:48.634492 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:48.634501 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:48.634559 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:48.634925 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:48.634926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:48.634928 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:48.634976 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.634978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.635793 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.635793 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.636961 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.636964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.637831 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.637832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.637905 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.637910 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:59:48.637957 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 22:59:48.637999 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":83,"thread_start_us":49,"threads_started":1}
I20260808 22:59:48.642174 26868 tablet_bootstrap.cc:492] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff: Bootstrap starting.
I20260808 22:59:48.642808 26869 tablet_bootstrap.cc:492] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9: Bootstrap starting.
I20260808 22:59:48.642853 26868 tablet_bootstrap.cc:654] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:48.642916 26736 tablet_service.cc:1467] Processing CreateTablet for tablet 7e6f3eb25c154f4dbae9f38cbfd32418 (DEFAULT_TABLE table=test-table [id=c1d8897c2e5546558ee1605463c5d56f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:48.643182 26868 log.cc:824] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:48.643253 26736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e6f3eb25c154f4dbae9f38cbfd32418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:48.643478 26869 tablet_bootstrap.cc:654] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:48.643733 26869 log.cc:824] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:48.643970 26868 tablet_bootstrap.cc:492] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff: No bootstrap required, opened a new log
I20260808 22:59:48.644021 26868 ts_tablet_manager.cc:1397] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:59:48.644454 26869 tablet_bootstrap.cc:492] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9: No bootstrap required, opened a new log
I20260808 22:59:48.644507 26869 ts_tablet_manager.cc:1397] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:59:48.645870 26872 tablet_bootstrap.cc:492] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d: Bootstrap starting.
I20260808 22:59:48.646596 26872 tablet_bootstrap.cc:654] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:48.646936 26868 raft_consensus.cc:348] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } }
I20260808 22:59:48.647114 26868 raft_consensus.cc:374] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:48.647197 26868 raft_consensus.cc:729] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a558ea4728844e59a1f7b787b2b1a7ff, State: Initialized, Role: FOLLOWER
I20260808 22:59:48.647351 26868 consensus_queue.cc:260] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } }
I20260808 22:59:48.647511 26872 log.cc:824] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:48.647732 26868 ts_tablet_manager.cc:1428] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260808 22:59:48.647775 26402 heartbeater.cc:503] Master 127.23.194.62:36133 was elected leader, sending a full tablet report...
I20260808 22:59:48.648351 26872 tablet_bootstrap.cc:492] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d: No bootstrap required, opened a new log
I20260808 22:59:48.648422 26872 ts_tablet_manager.cc:1397] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:59:48.648407 26869 raft_consensus.cc:348] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } }
I20260808 22:59:48.648550 26869 raft_consensus.cc:374] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:48.648583 26869 raft_consensus.cc:729] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 899db6005d564ab5aa986c4607a5abd9, State: Initialized, Role: FOLLOWER
I20260808 22:59:48.648809 26869 consensus_queue.cc:260] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } }
I20260808 22:59:48.649204 26869 ts_tablet_manager.cc:1428] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 22:59:48.649256 26612 heartbeater.cc:503] Master 127.23.194.62:36133 was elected leader, sending a full tablet report...
I20260808 22:59:48.651595 26872 raft_consensus.cc:348] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } }
I20260808 22:59:48.651717 26872 raft_consensus.cc:374] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:48.651737 26872 raft_consensus.cc:729] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba03fcd319324788b94267712f05276d, State: Initialized, Role: FOLLOWER
I20260808 22:59:48.651818 26872 consensus_queue.cc:260] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } }
I20260808 22:59:48.652103 26872 ts_tablet_manager.cc:1428] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260808 22:59:48.652205 26834 heartbeater.cc:503] Master 127.23.194.62:36133 was elected leader, sending a full tablet report...
W20260808 22:59:48.703064 26616 tablet.cc:2369] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:59:48.778407 26876 raft_consensus.cc:482] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:59:48.778568 26876 raft_consensus.cc:504] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } }
I20260808 22:59:48.779031 26876 leader_election.cc:288] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a558ea4728844e59a1f7b787b2b1a7ff (127.23.194.1:41423), ba03fcd319324788b94267712f05276d (127.23.194.3:33829)
I20260808 22:59:48.783304 26456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.779274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:41423 (local address 127.23.194.2:47299)
0808 22:59:48.779342 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:48.779356 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:48.779423 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:48.779584 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:48.779586 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:48.779588 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:48.779791 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.779794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.780833 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.780834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.782135 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.782138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.782922 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.782923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.783127 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.783132 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:59:48.783184 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 22:59:48.783226 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":44}
I20260808 22:59:48.783476 26879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.779416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:33829 (local address 127.23.194.2:54243)
0808 22:59:48.779702 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:48.779712 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:48.779756 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:48.779896 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:48.779897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:48.779898 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:48.780075 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.780077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.780942 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.780942 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.782240 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.782242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.782965 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.782966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.783222 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.783227 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:59:48.783319 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 22:59:48.783365 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":162,"threads_started":1}
I20260808 22:59:48.783819 26867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.779668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:54243 (local address 127.23.194.3:33829)
0808 22:59:48.779735 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:48.779737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:48.779749 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:59:48.779803 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:48.779804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:48.779812 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:48.779861 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:48.780189 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.780879 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.782301 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.782876 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.782935 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.782943 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:48.783714 (+ 771us) server_negotiation.cc:299] Negotiation successful
0808 22:59:48.783760 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:59:48.783842 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.779329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:47299 (local address 127.23.194.1:41423)
0808 22:59:48.779394 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:48.779396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:48.779405 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:59:48.779487 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:48.779488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:48.779498 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:48.779545 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:48.779879 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.780752 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.782207 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.782782 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.782878 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.782888 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:48.783746 (+ 858us) server_negotiation.cc:299] Negotiation successful
0808 22:59:48.783787 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:59:48.784122 26317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e6f3eb25c154f4dbae9f38cbfd32418" candidate_uuid: "899db6005d564ab5aa986c4607a5abd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" is_pre_election: true
I20260808 22:59:48.784168 26767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e6f3eb25c154f4dbae9f38cbfd32418" candidate_uuid: "899db6005d564ab5aa986c4607a5abd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba03fcd319324788b94267712f05276d" is_pre_election: true
I20260808 22:59:48.784269 26317 raft_consensus.cc:2454] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 899db6005d564ab5aa986c4607a5abd9 in term 0.
I20260808 22:59:48.784307 26767 raft_consensus.cc:2454] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 899db6005d564ab5aa986c4607a5abd9 in term 0.
I20260808 22:59:48.784508 26441 leader_election.cc:302] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 899db6005d564ab5aa986c4607a5abd9, a558ea4728844e59a1f7b787b2b1a7ff; no voters:
I20260808 22:59:48.784684 26876 raft_consensus.cc:2781] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:59:48.784750 26876 raft_consensus.cc:482] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:59:48.784767 26876 raft_consensus.cc:3037] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:48.786232 26876 raft_consensus.cc:504] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } }
I20260808 22:59:48.786356 26876 leader_election.cc:288] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [CANDIDATE]: Term 1 election: Requested vote from peers a558ea4728844e59a1f7b787b2b1a7ff (127.23.194.1:41423), ba03fcd319324788b94267712f05276d (127.23.194.3:33829)
I20260808 22:59:48.786517 26317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e6f3eb25c154f4dbae9f38cbfd32418" candidate_uuid: "899db6005d564ab5aa986c4607a5abd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a558ea4728844e59a1f7b787b2b1a7ff"
I20260808 22:59:48.786542 26767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e6f3eb25c154f4dbae9f38cbfd32418" candidate_uuid: "899db6005d564ab5aa986c4607a5abd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba03fcd319324788b94267712f05276d"
I20260808 22:59:48.786607 26767 raft_consensus.cc:3037] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:48.786607 26317 raft_consensus.cc:3037] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:48.787989 26317 raft_consensus.cc:2454] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 899db6005d564ab5aa986c4607a5abd9 in term 1.
I20260808 22:59:48.788028 26767 raft_consensus.cc:2454] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 899db6005d564ab5aa986c4607a5abd9 in term 1.
I20260808 22:59:48.788168 26441 leader_election.cc:302] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 899db6005d564ab5aa986c4607a5abd9, ba03fcd319324788b94267712f05276d; no voters:
I20260808 22:59:48.788266 26876 raft_consensus.cc:2781] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:48.788439 26876 raft_consensus.cc:686] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [term 1 LEADER]: Becoming Leader. State: Replica: 899db6005d564ab5aa986c4607a5abd9, State: Running, Role: LEADER
I20260808 22:59:48.788530 26876 consensus_queue.cc:237] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } }
I20260808 22:59:48.789463 26070 catalog_manager.cc:5495] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 899db6005d564ab5aa986c4607a5abd9 (127.23.194.2). New cstate: current_term: 1 leader_uuid: "899db6005d564ab5aa986c4607a5abd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } health_report { overall_health: HEALTHY } } }
W20260808 22:59:48.804915 26404 tablet.cc:2369] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:59:48.821892 26859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.819433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52078 (local address 127.23.194.2:45007)
0808 22:59:48.819497 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:48.819499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:48.819509 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:59:48.819576 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:48.819577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:48.819597 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:48.819654 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:48.819786 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.820249 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.821006 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.821199 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.821254 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.821467 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:48.821566 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:48.821635 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:48.821705 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 22:59:48.821736 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:59:48.823843 26317 raft_consensus.cc:1261] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff [term 1 FOLLOWER]: Refusing update from remote peer 899db6005d564ab5aa986c4607a5abd9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:48.824198 26880 consensus_queue.cc:1035] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:48.825558 26767 raft_consensus.cc:1261] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d [term 1 FOLLOWER]: Refusing update from remote peer 899db6005d564ab5aa986c4607a5abd9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:48.825805 26876 consensus_queue.cc:1035] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:48.828969 26889 mvcc.cc:205] Tried to move back new op lower bound from 7316398034219859968 to 7316398034078347264. Current Snapshot: MvccSnapshot[applied={T|T < 7316398034219859968 or (T in {7316398034219859968})}]
I20260808 22:59:48.829608 26894 mvcc.cc:205] Tried to move back new op lower bound from 7316398034219859968 to 7316398034078347264. Current Snapshot: MvccSnapshot[applied={T|T < 7316398034219859968}]
I20260808 22:59:48.830928 26070 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:59:48.831053 26070 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:59:48.831816 26070 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364: Sending DeleteTablet for 3 replicas of tablet 7e6f3eb25c154f4dbae9f38cbfd32418
I20260808 22:59:48.832286 26511 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e6f3eb25c154f4dbae9f38cbfd32418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:48 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:52062
I20260808 22:59:48.832521 26070 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47272:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:59:48.832623 26070 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:59:48.832621 26898 tablet_replica.cc:330] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9: stopping tablet replica
I20260808 22:59:48.832723 26898 raft_consensus.cc:2229] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:59:48.832810 26898 raft_consensus.cc:2258] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:48.833639 26736 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e6f3eb25c154f4dbae9f38cbfd32418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:48 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:53870
I20260808 22:59:48.833760 26898 ts_tablet_manager.cc:1905] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:59:48.835302 26511 tablet_service.cc:1467] Processing CreateTablet for tablet 24e6bf77fca04972bb839449025f0e2d (DEFAULT_TABLE table=test-table [id=eec4bb5f92464270a50245b47c0f3f0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:48.835418 26511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24e6bf77fca04972bb839449025f0e2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:48.835757 26898 ts_tablet_manager.cc:1918] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:59:48.835830 26898 log.cc:1196] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/7e6f3eb25c154f4dbae9f38cbfd32418
I20260808 22:59:48.836095 26898 ts_tablet_manager.cc:1939] T 7e6f3eb25c154f4dbae9f38cbfd32418 P 899db6005d564ab5aa986c4607a5abd9: Deleting consensus metadata
I20260808 22:59:48.836486 26050 catalog_manager.cc:4841] TS 899db6005d564ab5aa986c4607a5abd9 (127.23.194.2:45007): tablet 7e6f3eb25c154f4dbae9f38cbfd32418 (table test-table [id=c1d8897c2e5546558ee1605463c5d56f]) successfully deleted
I20260808 22:59:48.836856 26869 tablet_bootstrap.cc:492] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9: Bootstrap starting.
I20260808 22:59:48.836949 26283 tablet_service.cc:1467] Processing CreateTablet for tablet 24e6bf77fca04972bb839449025f0e2d (DEFAULT_TABLE table=test-table [id=eec4bb5f92464270a50245b47c0f3f0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:48.837059 26283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24e6bf77fca04972bb839449025f0e2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:48.837503 26900 tablet_replica.cc:330] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d: stopping tablet replica
I20260808 22:59:48.837560 26869 tablet_bootstrap.cc:654] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:48.837603 26900 raft_consensus.cc:2229] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:59:48.837718 26900 raft_consensus.cc:2258] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:48.837734 26736 tablet_service.cc:1467] Processing CreateTablet for tablet 24e6bf77fca04972bb839449025f0e2d (DEFAULT_TABLE table=test-table [id=eec4bb5f92464270a50245b47c0f3f0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:48.838142 26736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24e6bf77fca04972bb839449025f0e2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:48.838687 26869 tablet_bootstrap.cc:492] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9: No bootstrap required, opened a new log
I20260808 22:59:48.838744 26869 ts_tablet_manager.cc:1397] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:59:48.839112 26869 raft_consensus.cc:348] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } }
I20260808 22:59:48.839190 26869 raft_consensus.cc:374] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:48.839210 26869 raft_consensus.cc:729] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 899db6005d564ab5aa986c4607a5abd9, State: Initialized, Role: FOLLOWER
I20260808 22:59:48.839253 26869 consensus_queue.cc:260] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } }
I20260808 22:59:48.839341 26869 ts_tablet_manager.cc:1428] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:48.839651 26872 tablet_bootstrap.cc:492] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d: Bootstrap starting.
I20260808 22:59:48.840440 26872 tablet_bootstrap.cc:654] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:48.841290 26868 tablet_bootstrap.cc:492] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff: Bootstrap starting.
I20260808 22:59:48.841455 26286 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e6f3eb25c154f4dbae9f38cbfd32418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:48 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:37312
I20260808 22:59:48.841655 26905 tablet_replica.cc:330] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff: stopping tablet replica
I20260808 22:59:48.841773 26905 raft_consensus.cc:2229] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:59:48.841867 26905 raft_consensus.cc:2258] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:48.841910 26900 ts_tablet_manager.cc:1905] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:59:48.842272 26868 tablet_bootstrap.cc:654] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:48.843336 26872 tablet_bootstrap.cc:492] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d: No bootstrap required, opened a new log
I20260808 22:59:48.843394 26872 ts_tablet_manager.cc:1397] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260808 22:59:48.843650 26872 raft_consensus.cc:348] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } }
I20260808 22:59:48.843727 26872 raft_consensus.cc:374] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:48.843750 26872 raft_consensus.cc:729] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba03fcd319324788b94267712f05276d, State: Initialized, Role: FOLLOWER
I20260808 22:59:48.843799 26872 consensus_queue.cc:260] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } }
I20260808 22:59:48.843890 26872 ts_tablet_manager.cc:1428] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:48.844041 26900 ts_tablet_manager.cc:1918] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:59:48.844102 26900 log.cc:1196] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/7e6f3eb25c154f4dbae9f38cbfd32418
I20260808 22:59:48.844367 26900 ts_tablet_manager.cc:1939] T 7e6f3eb25c154f4dbae9f38cbfd32418 P ba03fcd319324788b94267712f05276d: Deleting consensus metadata
I20260808 22:59:48.844750 26868 tablet_bootstrap.cc:492] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff: No bootstrap required, opened a new log
I20260808 22:59:48.844803 26868 ts_tablet_manager.cc:1397] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 22:59:48.844806 26047 catalog_manager.cc:4841] TS ba03fcd319324788b94267712f05276d (127.23.194.3:33829): tablet 7e6f3eb25c154f4dbae9f38cbfd32418 (table test-table [id=c1d8897c2e5546558ee1605463c5d56f]) successfully deleted
I20260808 22:59:48.845248 26905 ts_tablet_manager.cc:1905] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:59:48.845302 26868 raft_consensus.cc:348] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } }
I20260808 22:59:48.845371 26868 raft_consensus.cc:374] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:48.845427 26868 raft_consensus.cc:729] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a558ea4728844e59a1f7b787b2b1a7ff, State: Initialized, Role: FOLLOWER
I20260808 22:59:48.845474 26868 consensus_queue.cc:260] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } }
I20260808 22:59:48.845628 26868 ts_tablet_manager.cc:1428] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:48.847301 26905 ts_tablet_manager.cc:1918] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:59:48.847378 26905 log.cc:1196] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-0/wal/wals/7e6f3eb25c154f4dbae9f38cbfd32418
I20260808 22:59:48.847757 26905 ts_tablet_manager.cc:1939] T 7e6f3eb25c154f4dbae9f38cbfd32418 P a558ea4728844e59a1f7b787b2b1a7ff: Deleting consensus metadata
I20260808 22:59:48.848198 26047 catalog_manager.cc:4841] TS a558ea4728844e59a1f7b787b2b1a7ff (127.23.194.1:41423): tablet 7e6f3eb25c154f4dbae9f38cbfd32418 (table test-table [id=c1d8897c2e5546558ee1605463c5d56f]) successfully deleted
W20260808 22:59:48.853035 26838 tablet.cc:2369] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:59:48.912864 26874 raft_consensus.cc:482] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:59:48.912972 26874 raft_consensus.cc:504] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } }
I20260808 22:59:48.913298 26874 leader_election.cc:288] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 899db6005d564ab5aa986c4607a5abd9 (127.23.194.2:45007), ba03fcd319324788b94267712f05276d (127.23.194.3:33829)
I20260808 22:59:48.913430 26878 raft_consensus.cc:482] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:59:48.913501 26878 raft_consensus.cc:504] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } }
I20260808 22:59:48.913834 26878 leader_election.cc:288] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a558ea4728844e59a1f7b787b2b1a7ff (127.23.194.1:41423), 899db6005d564ab5aa986c4607a5abd9 (127.23.194.2:45007)
I20260808 22:59:48.917801 26672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.914012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:41423 (local address 127.23.194.3:52871)
0808 22:59:48.914188 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:48.914199 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:48.914234 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:48.914520 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:48.914521 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:48.914523 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:48.914590 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.914592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.915541 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.915543 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.916762 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.916766 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.917522 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.917523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.917613 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.917617 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:59:48.917682 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 22:59:48.917725 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":62,"threads_started":1}
I20260808 22:59:48.918193 26859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.913737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:47327 (local address 127.23.194.2:45007)
0808 22:59:48.913787 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:48.913789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:48.914441 (+ 652us) server_negotiation.cc:407] Connection header received
0808 22:59:48.914515 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:48.914516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:48.914525 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:48.914555 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:48.914812 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.915478 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.916947 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.917581 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.917651 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.917658 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:48.918085 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 22:59:48.918126 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 22:59:48.918313 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.914082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:52871 (local address 127.23.194.1:41423)
0808 22:59:48.914120 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:48.914122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:48.914382 (+ 260us) server_negotiation.cc:407] Connection header received
0808 22:59:48.914408 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:48.914408 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:48.914417 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:48.914463 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:48.914666 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.915425 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.916864 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.917405 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.917476 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.917488 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:48.918209 (+ 721us) server_negotiation.cc:299] Negotiation successful
0808 22:59:48.918252 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 22:59:48.918494 26317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e6bf77fca04972bb839449025f0e2d" candidate_uuid: "ba03fcd319324788b94267712f05276d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" is_pre_election: true
I20260808 22:59:48.918586 26317 raft_consensus.cc:2454] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba03fcd319324788b94267712f05276d in term 0.
I20260808 22:59:48.918859 26909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.913551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:45007 (local address 127.23.194.1:47327)
0808 22:59:48.914258 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:48.914266 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:48.914441 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:48.914624 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:48.914625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:48.914627 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:48.914734 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.914736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.915581 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.915581 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.916850 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.916853 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.917720 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.917721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.917791 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.917795 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:59:48.918488 (+ 693us) client_negotiation.cc:240] Negotiation successful
0808 22:59:48.918536 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"thread_start_us":58,"threads_started":1}
I20260808 22:59:48.918869 26660 leader_election.cc:302] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a558ea4728844e59a1f7b787b2b1a7ff, ba03fcd319324788b94267712f05276d; no voters:
I20260808 22:59:48.918990 26878 raft_consensus.cc:2781] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:59:48.919027 26878 raft_consensus.cc:482] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:59:48.919047 26878 raft_consensus.cc:3037] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:48.919075 26911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.913949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:45007 (local address 127.23.194.3:35685)
0808 22:59:48.914170 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:48.914179 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:48.914368 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:48.914783 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:48.914784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:48.914786 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:48.915067 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.915070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.916275 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.916276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.917754 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.917757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.918685 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.918686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.918840 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.918847 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 22:59:48.918885 (+ 38us) client_negotiation.cc:240] Negotiation successful
0808 22:59:48.918923 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198}
I20260808 22:59:48.919445 26547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e6bf77fca04972bb839449025f0e2d" candidate_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "899db6005d564ab5aa986c4607a5abd9" is_pre_election: true
I20260808 22:59:48.919627 26547 raft_consensus.cc:2454] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a558ea4728844e59a1f7b787b2b1a7ff in term 0.
I20260808 22:59:48.919961 26218 leader_election.cc:302] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 899db6005d564ab5aa986c4607a5abd9, a558ea4728844e59a1f7b787b2b1a7ff; no voters:
I20260808 22:59:48.920091 26874 raft_consensus.cc:2781] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:59:48.920123 26874 raft_consensus.cc:482] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:59:48.920142 26874 raft_consensus.cc:3037] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:48.920233 26878 raft_consensus.cc:504] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } }
I20260808 22:59:48.920365 26878 leader_election.cc:288] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [CANDIDATE]: Term 1 election: Requested vote from peers a558ea4728844e59a1f7b787b2b1a7ff (127.23.194.1:41423), 899db6005d564ab5aa986c4607a5abd9 (127.23.194.2:45007)
I20260808 22:59:48.920569 26317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e6bf77fca04972bb839449025f0e2d" candidate_uuid: "ba03fcd319324788b94267712f05276d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a558ea4728844e59a1f7b787b2b1a7ff"
I20260808 22:59:48.920953 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.914297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:35685 (local address 127.23.194.2:45007)
0808 22:59:48.914491 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:48.914492 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:48.914497 (+ 5us) server_negotiation.cc:407] Connection header received
0808 22:59:48.914515 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:48.914516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:48.914527 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:48.915143 (+ 616us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:48.915186 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.916178 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.917841 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.918585 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.919293 (+ 708us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.919305 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:48.919326 (+ 21us) server_negotiation.cc:299] Negotiation successful
0808 22:59:48.919359 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":62,"threads_started":1}
I20260808 22:59:48.921314 26546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e6bf77fca04972bb839449025f0e2d" candidate_uuid: "ba03fcd319324788b94267712f05276d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "899db6005d564ab5aa986c4607a5abd9"
I20260808 22:59:48.921420 26546 raft_consensus.cc:3037] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:48.921353 26874 raft_consensus.cc:504] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } }
I20260808 22:59:48.921523 26874 leader_election.cc:288] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff [CANDIDATE]: Term 1 election: Requested vote from peers 899db6005d564ab5aa986c4607a5abd9 (127.23.194.2:45007), ba03fcd319324788b94267712f05276d (127.23.194.3:33829)
I20260808 22:59:48.921613 26317 raft_consensus.cc:2379] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ba03fcd319324788b94267712f05276d in current term 1: Already voted for candidate a558ea4728844e59a1f7b787b2b1a7ff in this term.
I20260808 22:59:48.922339 26544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e6bf77fca04972bb839449025f0e2d" candidate_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "899db6005d564ab5aa986c4607a5abd9"
I20260808 22:59:48.922597 26546 raft_consensus.cc:2454] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba03fcd319324788b94267712f05276d in term 1.
I20260808 22:59:48.922854 26547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e6bf77fca04972bb839449025f0e2d" candidate_uuid: "ba03fcd319324788b94267712f05276d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "899db6005d564ab5aa986c4607a5abd9" is_pre_election: true
I20260808 22:59:48.922930 26547 raft_consensus.cc:2362] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ba03fcd319324788b94267712f05276d in term 1. Re-sending same reply.
I20260808 22:59:48.922945 26664 leader_election.cc:302] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 899db6005d564ab5aa986c4607a5abd9, ba03fcd319324788b94267712f05276d; no voters: a558ea4728844e59a1f7b787b2b1a7ff
I20260808 22:59:48.923038 26867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.913831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:46157 (local address 127.23.194.3:33829)
0808 22:59:48.914000 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:48.914002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:48.914010 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:59:48.914040 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:48.914041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:48.914050 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:48.914102 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:48.919955 (+ 5853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.920580 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.922029 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.922504 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.922552 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.922561 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:48.922951 (+ 390us) server_negotiation.cc:299] Negotiation successful
0808 22:59:48.922989 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260808 22:59:48.923197 26878 raft_consensus.cc:2781] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:48.923282 26908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.913530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:33829 (local address 127.23.194.1:46157)
0808 22:59:48.913739 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:48.913751 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:48.913956 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:48.919728 (+ 5772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:48.919729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:48.919731 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:48.919863 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.919865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.920684 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.920685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.921940 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.921942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.922741 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.922743 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.922844 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.922850 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:59:48.923113 (+ 263us) client_negotiation.cc:240] Negotiation successful
0808 22:59:48.923161 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":88,"threads_started":1}
I20260808 22:59:48.923403 26878 raft_consensus.cc:686] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [term 1 LEADER]: Becoming Leader. State: Replica: ba03fcd319324788b94267712f05276d, State: Running, Role: LEADER
I20260808 22:59:48.923480 26878 consensus_queue.cc:237] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } }
I20260808 22:59:48.923871 26767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e6bf77fca04972bb839449025f0e2d" candidate_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba03fcd319324788b94267712f05276d" is_pre_election: true
I20260808 22:59:48.923924 26766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e6bf77fca04972bb839449025f0e2d" candidate_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba03fcd319324788b94267712f05276d"
I20260808 22:59:48.924278 26215 leader_election.cc:302] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a558ea4728844e59a1f7b787b2b1a7ff; no voters: 899db6005d564ab5aa986c4607a5abd9, ba03fcd319324788b94267712f05276d
I20260808 22:59:48.924328 26070 catalog_manager.cc:5495] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d reported cstate change: term changed from 0 to 1, leader changed from <none> to ba03fcd319324788b94267712f05276d (127.23.194.3). New cstate: current_term: 1 leader_uuid: "ba03fcd319324788b94267712f05276d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } health_report { overall_health: HEALTHY } } }
I20260808 22:59:48.924443 26874 raft_consensus.cc:2726] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
Aug 08 22:59:48 dist-test-slave-f0tn krb5kdc[25998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229988, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:48.973873 26922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.966202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:36133 (local address 127.0.0.1:47274)
0808 22:59:48.966494 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:48.966505 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:48.966678 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:48.966832 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:48.966833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:48.967253 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:48.967348 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.967351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.968021 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.968022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.968565 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.968567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.968880 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.968881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.968931 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.969531 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:48.969536 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:48.971051 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:48.972739 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:48.972748 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:48.972750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:48.972948 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:48.973151 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:48.973152 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:48.973152 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:48.973213 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:48.973482 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:48.973491 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:48.973547 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 22:59:48.973612 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 22:59:48.973740 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":174,"threads_started":1}
I20260808 22:59:48.973883 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.966299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47274 (local address 127.23.194.62:36133)
0808 22:59:48.966364 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:48.966366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:48.966671 (+ 305us) server_negotiation.cc:407] Connection header received
0808 22:59:48.966745 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:48.966746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:48.966759 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:48.966810 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:48.967439 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.967931 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.968640 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.968816 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.968877 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.971135 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:48.971137 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:48.971137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:48.971145 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:48.972660 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:48.973017 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:48.973018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:48.973018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:48.973076 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:48.973266 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:48.973267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:48.973267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:48.973425 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:48.973498 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:48.973652 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 22:59:48.973804 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":83,"server-negotiator.queue_time_us":32}
I20260808 22:59:48.976913 26070 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:47274:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 22:59:48.977197 26070 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:59:48.978706 26511 tablet_service.cc:1467] Processing CreateTablet for tablet 8ad28c209ff147c1936ea3d2b067d183 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3223b941f8754bed807763cb3e12b7da]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:59:48.978771 26286 tablet_service.cc:1467] Processing CreateTablet for tablet 8ad28c209ff147c1936ea3d2b067d183 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3223b941f8754bed807763cb3e12b7da]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:59:48.978930 26286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ad28c209ff147c1936ea3d2b067d183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:48.978935 26511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ad28c209ff147c1936ea3d2b067d183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:48.980567 26736 tablet_service.cc:1467] Processing CreateTablet for tablet 8ad28c209ff147c1936ea3d2b067d183 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3223b941f8754bed807763cb3e12b7da]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:59:48.980726 26736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ad28c209ff147c1936ea3d2b067d183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:48.981473 26869 tablet_bootstrap.cc:492] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9: Bootstrap starting.
I20260808 22:59:48.981953 26869 tablet_bootstrap.cc:654] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:48.982667 26872 tablet_bootstrap.cc:492] T 8ad28c209ff147c1936ea3d2b067d183 P ba03fcd319324788b94267712f05276d: Bootstrap starting.
I20260808 22:59:48.983031 26868 tablet_bootstrap.cc:492] T 8ad28c209ff147c1936ea3d2b067d183 P a558ea4728844e59a1f7b787b2b1a7ff: Bootstrap starting.
I20260808 22:59:48.983213 26869 tablet_bootstrap.cc:492] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9: No bootstrap required, opened a new log
I20260808 22:59:48.983255 26869 ts_tablet_manager.cc:1397] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:59:48.983407 26869 raft_consensus.cc:348] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } }
I20260808 22:59:48.983490 26869 raft_consensus.cc:374] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:48.983513 26869 raft_consensus.cc:729] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 899db6005d564ab5aa986c4607a5abd9, State: Initialized, Role: FOLLOWER
I20260808 22:59:48.983520 26872 tablet_bootstrap.cc:654] T 8ad28c209ff147c1936ea3d2b067d183 P ba03fcd319324788b94267712f05276d: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:48.983563 26869 consensus_queue.cc:260] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } }
I20260808 22:59:48.983659 26869 ts_tablet_manager.cc:1428] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 22:59:48.984018 26868 tablet_bootstrap.cc:654] T 8ad28c209ff147c1936ea3d2b067d183 P a558ea4728844e59a1f7b787b2b1a7ff: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:48.984747 26876 tablet_replica.cc:439] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } } }
I20260808 22:59:48.984829 26876 tablet_replica.cc:442] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:59:48.985307 26872 tablet_bootstrap.cc:492] T 8ad28c209ff147c1936ea3d2b067d183 P ba03fcd319324788b94267712f05276d: No bootstrap required, opened a new log
I20260808 22:59:48.985359 26872 ts_tablet_manager.cc:1397] T 8ad28c209ff147c1936ea3d2b067d183 P ba03fcd319324788b94267712f05276d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 22:59:48.985700 26872 raft_consensus.cc:348] T 8ad28c209ff147c1936ea3d2b067d183 P ba03fcd319324788b94267712f05276d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } }
I20260808 22:59:48.985778 26872 raft_consensus.cc:374] T 8ad28c209ff147c1936ea3d2b067d183 P ba03fcd319324788b94267712f05276d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:48.985797 26872 raft_consensus.cc:729] T 8ad28c209ff147c1936ea3d2b067d183 P ba03fcd319324788b94267712f05276d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba03fcd319324788b94267712f05276d, State: Initialized, Role: FOLLOWER
I20260808 22:59:48.985837 26872 consensus_queue.cc:260] T 8ad28c209ff147c1936ea3d2b067d183 P ba03fcd319324788b94267712f05276d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } }
I20260808 22:59:48.985850 26868 tablet_bootstrap.cc:492] T 8ad28c209ff147c1936ea3d2b067d183 P a558ea4728844e59a1f7b787b2b1a7ff: No bootstrap required, opened a new log
I20260808 22:59:48.985896 26868 ts_tablet_manager.cc:1397] T 8ad28c209ff147c1936ea3d2b067d183 P a558ea4728844e59a1f7b787b2b1a7ff: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:59:48.985921 26872 ts_tablet_manager.cc:1428] T 8ad28c209ff147c1936ea3d2b067d183 P ba03fcd319324788b94267712f05276d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 22:59:48.986184 26868 raft_consensus.cc:348] T 8ad28c209ff147c1936ea3d2b067d183 P a558ea4728844e59a1f7b787b2b1a7ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } }
I20260808 22:59:48.986253 26868 raft_consensus.cc:374] T 8ad28c209ff147c1936ea3d2b067d183 P a558ea4728844e59a1f7b787b2b1a7ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:48.986270 26868 raft_consensus.cc:729] T 8ad28c209ff147c1936ea3d2b067d183 P a558ea4728844e59a1f7b787b2b1a7ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a558ea4728844e59a1f7b787b2b1a7ff, State: Initialized, Role: FOLLOWER
I20260808 22:59:48.986310 26868 consensus_queue.cc:260] T 8ad28c209ff147c1936ea3d2b067d183 P a558ea4728844e59a1f7b787b2b1a7ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } }
I20260808 22:59:48.986399 26868 ts_tablet_manager.cc:1428] T 8ad28c209ff147c1936ea3d2b067d183 P a558ea4728844e59a1f7b787b2b1a7ff: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:48.986469 26874 tablet_replica.cc:439] T 8ad28c209ff147c1936ea3d2b067d183 P a558ea4728844e59a1f7b787b2b1a7ff: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } } }
I20260808 22:59:48.986558 26874 tablet_replica.cc:442] T 8ad28c209ff147c1936ea3d2b067d183 P a558ea4728844e59a1f7b787b2b1a7ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:59:48.986533 26878 tablet_replica.cc:439] T 8ad28c209ff147c1936ea3d2b067d183 P ba03fcd319324788b94267712f05276d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } } }
I20260808 22:59:48.986627 26878 tablet_replica.cc:442] T 8ad28c209ff147c1936ea3d2b067d183 P ba03fcd319324788b94267712f05276d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:59:48.987555 26876 raft_consensus.cc:482] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:59:48.987610 26876 raft_consensus.cc:504] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } }
I20260808 22:59:48.987720 26876 leader_election.cc:288] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a558ea4728844e59a1f7b787b2b1a7ff (127.23.194.1:41423), ba03fcd319324788b94267712f05276d (127.23.194.3:33829)
I20260808 22:59:48.987879 26317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ad28c209ff147c1936ea3d2b067d183" candidate_uuid: "899db6005d564ab5aa986c4607a5abd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" is_pre_election: true
I20260808 22:59:48.987879 26766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ad28c209ff147c1936ea3d2b067d183" candidate_uuid: "899db6005d564ab5aa986c4607a5abd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba03fcd319324788b94267712f05276d" is_pre_election: true
I20260808 22:59:48.987947 26317 raft_consensus.cc:2454] T 8ad28c209ff147c1936ea3d2b067d183 P a558ea4728844e59a1f7b787b2b1a7ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 899db6005d564ab5aa986c4607a5abd9 in term 0.
I20260808 22:59:48.987947 26766 raft_consensus.cc:2454] T 8ad28c209ff147c1936ea3d2b067d183 P ba03fcd319324788b94267712f05276d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 899db6005d564ab5aa986c4607a5abd9 in term 0.
I20260808 22:59:48.988075 26441 leader_election.cc:302] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 899db6005d564ab5aa986c4607a5abd9, ba03fcd319324788b94267712f05276d; no voters:
I20260808 22:59:48.988179 26876 raft_consensus.cc:2781] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:59:48.988221 26876 raft_consensus.cc:482] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:59:48.988245 26876 raft_consensus.cc:3037] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:48.988910 26876 raft_consensus.cc:504] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } }
I20260808 22:59:48.989008 26876 leader_election.cc:288] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9 [CANDIDATE]: Term 1 election: Requested vote from peers a558ea4728844e59a1f7b787b2b1a7ff (127.23.194.1:41423), ba03fcd319324788b94267712f05276d (127.23.194.3:33829)
I20260808 22:59:48.989219 26766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ad28c209ff147c1936ea3d2b067d183" candidate_uuid: "899db6005d564ab5aa986c4607a5abd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba03fcd319324788b94267712f05276d"
I20260808 22:59:48.989228 26317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ad28c209ff147c1936ea3d2b067d183" candidate_uuid: "899db6005d564ab5aa986c4607a5abd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a558ea4728844e59a1f7b787b2b1a7ff"
I20260808 22:59:48.989287 26766 raft_consensus.cc:3037] T 8ad28c209ff147c1936ea3d2b067d183 P ba03fcd319324788b94267712f05276d [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:48.989288 26317 raft_consensus.cc:3037] T 8ad28c209ff147c1936ea3d2b067d183 P a558ea4728844e59a1f7b787b2b1a7ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:48.990031 26317 raft_consensus.cc:2454] T 8ad28c209ff147c1936ea3d2b067d183 P a558ea4728844e59a1f7b787b2b1a7ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 899db6005d564ab5aa986c4607a5abd9 in term 1.
I20260808 22:59:48.990170 26441 leader_election.cc:302] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 899db6005d564ab5aa986c4607a5abd9, a558ea4728844e59a1f7b787b2b1a7ff; no voters:
I20260808 22:59:48.990221 26766 raft_consensus.cc:2454] T 8ad28c209ff147c1936ea3d2b067d183 P ba03fcd319324788b94267712f05276d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 899db6005d564ab5aa986c4607a5abd9 in term 1.
I20260808 22:59:48.990276 26876 raft_consensus.cc:2781] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:48.990340 26876 raft_consensus.cc:686] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9 [term 1 LEADER]: Becoming Leader. State: Replica: 899db6005d564ab5aa986c4607a5abd9, State: Running, Role: LEADER
I20260808 22:59:48.990387 26876 consensus_queue.cc:237] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } }
I20260808 22:59:48.990634 26880 tablet_replica.cc:439] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9: TxnStatusTablet state changed. Reason: New leader 899db6005d564ab5aa986c4607a5abd9. Latest consensus state: current_term: 1 leader_uuid: "899db6005d564ab5aa986c4607a5abd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } } }
I20260808 22:59:48.990711 26880 tablet_replica.cc:442] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:59:48.990938 26066 catalog_manager.cc:5495] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 899db6005d564ab5aa986c4607a5abd9 (127.23.194.2). New cstate: current_term: 1 leader_uuid: "899db6005d564ab5aa986c4607a5abd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } health_report { overall_health: UNKNOWN } } }
I20260808 22:59:48.991003 26933 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 22:59:48.991097 26933 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 22:59:48.998867 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.996257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52082 (local address 127.23.194.2:45007)
0808 22:59:48.996306 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:48.996308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:48.996348 (+ 40us) server_negotiation.cc:407] Connection header received
0808 22:59:48.996430 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:48.996431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:48.996440 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:48.996486 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:48.996642 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.997105 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.998197 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.998387 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.998446 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.998648 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:48.998678 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:48.998730 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:48.998793 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 22:59:48.998813 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 22:59:48.998917 26922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:48.996210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:45007 (local address 127.0.0.1:52082)
0808 22:59:48.996280 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:48.996293 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:48.996365 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:48.996494 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:48.996495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:48.996497 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:59:48.996557 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.996560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.997187 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.997188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.998120 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:48.998123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:48.998456 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:48.998457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:48.998533 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:48.998553 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:59:48.998740 (+ 187us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:59:48.998746 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:59:48.998788 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 22:59:48.998835 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 22:59:49.000257 26317 raft_consensus.cc:1261] T 8ad28c209ff147c1936ea3d2b067d183 P a558ea4728844e59a1f7b787b2b1a7ff [term 1 FOLLOWER]: Refusing update from remote peer 899db6005d564ab5aa986c4607a5abd9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:49.000257 26766 raft_consensus.cc:1261] T 8ad28c209ff147c1936ea3d2b067d183 P ba03fcd319324788b94267712f05276d [term 1 FOLLOWER]: Refusing update from remote peer 899db6005d564ab5aa986c4607a5abd9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:49.000465 26880 consensus_queue.cc:1035] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:49.000573 26876 consensus_queue.cc:1035] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:49.001261 26874 tablet_replica.cc:439] T 8ad28c209ff147c1936ea3d2b067d183 P a558ea4728844e59a1f7b787b2b1a7ff: TxnStatusTablet state changed. Reason: New leader 899db6005d564ab5aa986c4607a5abd9. Latest consensus state: current_term: 1 leader_uuid: "899db6005d564ab5aa986c4607a5abd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } } }
I20260808 22:59:49.001356 26874 tablet_replica.cc:442] T 8ad28c209ff147c1936ea3d2b067d183 P a558ea4728844e59a1f7b787b2b1a7ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:59:49.001422 26878 tablet_replica.cc:439] T 8ad28c209ff147c1936ea3d2b067d183 P ba03fcd319324788b94267712f05276d: TxnStatusTablet state changed. Reason: New leader 899db6005d564ab5aa986c4607a5abd9. Latest consensus state: current_term: 1 leader_uuid: "899db6005d564ab5aa986c4607a5abd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } } }
I20260808 22:59:49.001513 26878 tablet_replica.cc:442] T 8ad28c209ff147c1936ea3d2b067d183 P ba03fcd319324788b94267712f05276d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:59:49.002084 26876 tablet_replica.cc:439] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "899db6005d564ab5aa986c4607a5abd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } } }
I20260808 22:59:49.002162 26876 tablet_replica.cc:442] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:59:49.002280 26874 tablet_replica.cc:439] T 8ad28c209ff147c1936ea3d2b067d183 P a558ea4728844e59a1f7b787b2b1a7ff: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "899db6005d564ab5aa986c4607a5abd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } } }
I20260808 22:59:49.002364 26874 tablet_replica.cc:442] T 8ad28c209ff147c1936ea3d2b067d183 P a558ea4728844e59a1f7b787b2b1a7ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:59:49.002883 26887 mvcc.cc:205] Tried to move back new op lower bound from 7316398034943401984 to 7316398034904993792. Current Snapshot: MvccSnapshot[applied={T|T < 7316398034943401984 or (T in {7316398034943401984})}]
I20260808 22:59:49.006021 26880 tablet_replica.cc:439] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "899db6005d564ab5aa986c4607a5abd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } } }
I20260808 22:59:49.006111 26880 tablet_replica.cc:442] T 8ad28c209ff147c1936ea3d2b067d183 P 899db6005d564ab5aa986c4607a5abd9: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:59:49.006567 26878 tablet_replica.cc:439] T 8ad28c209ff147c1936ea3d2b067d183 P ba03fcd319324788b94267712f05276d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "899db6005d564ab5aa986c4607a5abd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 } } peers { permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 } } peers { permanent_uuid: "ba03fcd319324788b94267712f05276d" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33829 } } }
I20260808 22:59:49.006649 26878 tablet_replica.cc:442] T 8ad28c209ff147c1936ea3d2b067d183 P ba03fcd319324788b94267712f05276d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:59:49.007860 26867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:49.004833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53878 (local address 127.23.194.3:33829)
0808 22:59:49.004943 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:49.004945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:49.004954 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:59:49.004989 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:49.004990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:49.005011 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:49.005080 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:49.005172 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:49.005952 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:49.007032 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:49.007215 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:49.007269 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:49.007441 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:49.007572 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:49.007642 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:49.007691 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 22:59:49.007709 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 22:59:49.013298 26847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:49.010760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:45007 (local address 127.23.194.3:46457)
0808 22:59:49.010838 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:49.010850 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:49.010895 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:49.011179 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:49.011180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:49.011182 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:59:49.011320 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:49.011322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:49.011934 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:49.011935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:49.012501 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:49.012503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:49.012829 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:49.012830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:49.012945 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:49.012961 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:59:49.013130 (+ 169us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:59:49.013132 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 22:59:49.013185 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 22:59:49.013232 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 22:59:49.013286 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:49.010977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:46457 (local address 127.23.194.2:45007)
0808 22:59:49.011035 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:49.011037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:49.011044 (+ 7us) server_negotiation.cc:407] Connection header received
0808 22:59:49.011062 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:49.011062 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:49.011070 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:49.011160 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:49.011390 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:49.011859 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:49.012580 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:49.012767 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:49.012826 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:49.013044 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:49.013075 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:49.013123 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:49.013189 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 22:59:49.013212 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 22:59:49.016237 26547 raft_consensus.cc:1261] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9 [term 1 FOLLOWER]: Refusing update from remote peer ba03fcd319324788b94267712f05276d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:49.016410 26317 raft_consensus.cc:1261] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff [term 1 FOLLOWER]: Refusing update from remote peer ba03fcd319324788b94267712f05276d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:49.016651 26914 consensus_queue.cc:1035] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [LEADER]: Connected to new peer: Peer: permanent_uuid: "899db6005d564ab5aa986c4607a5abd9" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 45007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:49.016798 26878 consensus_queue.cc:1035] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a558ea4728844e59a1f7b787b2b1a7ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:49.026731 26867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:49.024014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:53637 (local address 127.23.194.3:33829)
0808 22:59:49.024055 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:49.024058 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:49.024210 (+ 152us) server_negotiation.cc:407] Connection header received
0808 22:59:49.024277 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:49.024277 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:49.024290 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:49.024345 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:49.024508 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:49.025006 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:49.025886 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:49.026154 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:49.026223 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:49.026444 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:49.026493 (+ 49us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:49.026527 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:49.026628 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 22:59:49.026655 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 22:59:49.026767 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:49.023780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:33829 (local address 127.23.194.2:53637)
0808 22:59:49.024022 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:49.024035 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:49.024221 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:49.024394 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:49.024395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:49.024397 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:59:49.024441 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:49.024442 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:49.025087 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:49.025088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:49.025800 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:49.025803 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:49.026262 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:49.026264 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:49.026326 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:49.026344 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:59:49.026574 (+ 230us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:59:49.026575 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 22:59:49.026618 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 22:59:49.026662 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":129,"threads_started":1}
I20260808 22:59:49.036854 26066 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:59:49.036975 26066 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:59:49.037633 26066 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c5b289853643455c896139aab7673364: Sending DeleteTablet for 3 replicas of tablet 24e6bf77fca04972bb839449025f0e2d
I20260808 22:59:49.037894 26736 tablet_service.cc:1514] Processing DeleteTablet for tablet 24e6bf77fca04972bb839449025f0e2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:49 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:53870
I20260808 22:59:49.038007 26900 tablet_replica.cc:330] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d: stopping tablet replica
I20260808 22:59:49.037997 26511 tablet_service.cc:1514] Processing DeleteTablet for tablet 24e6bf77fca04972bb839449025f0e2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:49 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:52062
I20260808 22:59:49.038064 26900 raft_consensus.cc:2229] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:59:49.038087 26898 tablet_replica.cc:330] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9: stopping tablet replica
I20260808 22:59:49.038129 26898 raft_consensus.cc:2229] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:59:49.038134 26900 raft_consensus.cc:2258] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:49.038161 26898 raft_consensus.cc:2258] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:49.038323 26286 tablet_service.cc:1514] Processing DeleteTablet for tablet 24e6bf77fca04972bb839449025f0e2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:49 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:37312
I20260808 22:59:49.038432 26905 tablet_replica.cc:330] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff: stopping tablet replica
I20260808 22:59:49.038493 26905 raft_consensus.cc:2229] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:59:49.038513 26898 ts_tablet_manager.cc:1905] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:59:49.038534 26905 raft_consensus.cc:2258] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:49.038602 26900 ts_tablet_manager.cc:1905] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:59:49.038818 26905 ts_tablet_manager.cc:1905] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:59:49.040401 26898 ts_tablet_manager.cc:1918] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:59:49.040464 26898 log.cc:1196] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/24e6bf77fca04972bb839449025f0e2d
I20260808 22:59:49.040670 26898 ts_tablet_manager.cc:1939] T 24e6bf77fca04972bb839449025f0e2d P 899db6005d564ab5aa986c4607a5abd9: Deleting consensus metadata
I20260808 22:59:49.040663 26905 ts_tablet_manager.cc:1918] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:59:49.040726 26905 log.cc:1196] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-0/wal/wals/24e6bf77fca04972bb839449025f0e2d
I20260808 22:59:49.040901 26905 ts_tablet_manager.cc:1939] T 24e6bf77fca04972bb839449025f0e2d P a558ea4728844e59a1f7b787b2b1a7ff: Deleting consensus metadata
I20260808 22:59:49.040998 26900 ts_tablet_manager.cc:1918] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:59:49.041043 26900 log.cc:1196] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/24e6bf77fca04972bb839449025f0e2d
I20260808 22:59:49.041244 26900 ts_tablet_manager.cc:1939] T 24e6bf77fca04972bb839449025f0e2d P ba03fcd319324788b94267712f05276d: Deleting consensus metadata
I20260808 22:59:49.041288 26047 catalog_manager.cc:4841] TS a558ea4728844e59a1f7b787b2b1a7ff (127.23.194.1:41423): tablet 24e6bf77fca04972bb839449025f0e2d (table test-table [id=eec4bb5f92464270a50245b47c0f3f0c]) successfully deleted
I20260808 22:59:49.041534 26047 catalog_manager.cc:4841] TS ba03fcd319324788b94267712f05276d (127.23.194.3:33829): tablet 24e6bf77fca04972bb839449025f0e2d (table test-table [id=eec4bb5f92464270a50245b47c0f3f0c]) successfully deleted
I20260808 22:59:49.041601 26050 catalog_manager.cc:4841] TS 899db6005d564ab5aa986c4607a5abd9 (127.23.194.2:45007): tablet 24e6bf77fca04972bb839449025f0e2d (table test-table [id=eec4bb5f92464270a50245b47c0f3f0c]) successfully deleted
Aug 08 22:59:49 dist-test-slave-f0tn krb5kdc[25998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229988, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.194.1@KRBTEST.COM
I20260808 22:59:49.056589 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:49.042067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37316 (local address 127.23.194.1:41423)
0808 22:59:49.042108 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:49.042110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:49.045656 (+ 3546us) server_negotiation.cc:407] Connection header received
0808 22:59:49.045714 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:49.045715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:49.045729 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:49.045786 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:49.046672 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:49.047427 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:49.048536 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:49.048836 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:49.048902 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:49.052681 (+ 3779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:49.052693 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:49.052693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:49.052701 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:49.055252 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:49.055683 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:49.055687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:49.055688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:49.055775 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:49.055970 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:49.055971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:49.055972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:49.056144 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:49.056278 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:49.056356 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 22:59:49.056448 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 22:59:49.056828 26394 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37316
I20260808 22:59:49.065081 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:49.057965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47286 (local address 127.23.194.62:36133)
0808 22:59:49.058023 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:49.058025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:49.058033 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:59:49.058056 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:49.058057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:49.058068 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:49.058121 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:49.058725 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:49.059414 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:49.060420 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:49.060702 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:49.060788 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:49.062249 (+ 1461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:49.062250 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:49.062250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:49.062258 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:49.063959 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:49.064332 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:49.064333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:49.064333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:49.064393 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:49.064581 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:49.064581 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:49.064582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:49.064705 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:49.064794 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:49.064904 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 22:59:49.064983 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 22:59:49.065294 26066 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47286
I20260808 22:59:49.066395 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 26187
I20260808 22:59:49.084267 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 26412
I20260808 22:59:49.097536 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 26633
I20260808 22:59:49.104264 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 26018
2026-08-08T22:59:49Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2356 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:49 dist-test-slave-f0tn krb5kdc[26961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:49 dist-test-slave-f0tn krb5kdc[26961](info): set up 2 sockets
Aug 08 22:59:49 dist-test-slave-f0tn krb5kdc[26961](info): commencing operation
krb5kdc: starting...
W20260808 22:59:50.271183 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 22:59:50 dist-test-slave-f0tn krb5kdc[26961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 22:59:50.383556 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:45317
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:34647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:45317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:50.463865 26988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:50.464134 26988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:50.464167 26988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:50.465294 26988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:50.465333 26988 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 22:59:50.465350 26988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:50.465368 26988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:50.465413 26988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:50.465437 26988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:50.466909 26988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:34647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:45317
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:45317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:50.467166 26988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:50.467366 26988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:50.471902 26999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:50.471899 26995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:50.471926 26996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:50.472038 26988 server_base.cc:1034] running on GCE node
I20260808 22:59:50.472359 26988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:50.472618 26988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:50.473745 26988 hybrid_clock.cc:648] HybridClock initialized: now 1786229990473736 us; error 27 us; skew 500 ppm
Aug 08 22:59:50 dist-test-slave-f0tn krb5kdc[26961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229990, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:50.475958 26988 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 22:59:50.476383 26988 webserver.cc:460] Webserver started at http://127.23.194.62:40813/ using document root <none> and password file <none>
I20260808 22:59:50.476591 26988 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:50.476639 26988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:50.476754 26988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:50.477759 26988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "865550ce39cd438dba96f8cbe1863611"
format_stamp: "Formatted at 2026-08-08 22:59:50 on dist-test-slave-f0tn"
server_key: "4f0b14240380343c808bd4a31f72e123"
server_key_iv: "40fbbd1e625020a10727e249b30223a2"
server_key_version: "encryptionkey@0"
I20260808 22:59:50.478075 26988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "865550ce39cd438dba96f8cbe1863611"
format_stamp: "Formatted at 2026-08-08 22:59:50 on dist-test-slave-f0tn"
server_key: "4f0b14240380343c808bd4a31f72e123"
server_key_iv: "40fbbd1e625020a10727e249b30223a2"
server_key_version: "encryptionkey@0"
I20260808 22:59:50.479256 26988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:59:50.479878 27009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:50.480022 26988 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 22:59:50.480104 26988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "865550ce39cd438dba96f8cbe1863611"
format_stamp: "Formatted at 2026-08-08 22:59:50 on dist-test-slave-f0tn"
server_key: "4f0b14240380343c808bd4a31f72e123"
server_key_iv: "40fbbd1e625020a10727e249b30223a2"
server_key_version: "encryptionkey@0"
I20260808 22:59:50.480168 26988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:50.492199 26988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:50.493908 26988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:50.494032 26988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:50.498438 26988 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:45317
I20260808 22:59:50.498505 27100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:45317 every 8 connection(s)
I20260808 22:59:50.498787 26988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:50.499365 27101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:50.501510 27101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611: Bootstrap starting.
I20260808 22:59:50.502058 27101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:50.502329 27101 log.cc:824] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:50.502934 27101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611: No bootstrap required, opened a new log
I20260808 22:59:50.505080 27101 raft_consensus.cc:348] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865550ce39cd438dba96f8cbe1863611" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 45317 } }
I20260808 22:59:50.505214 27101 raft_consensus.cc:374] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:50.505236 27101 raft_consensus.cc:729] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 865550ce39cd438dba96f8cbe1863611, State: Initialized, Role: FOLLOWER
I20260808 22:59:50.505367 27101 consensus_queue.cc:260] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865550ce39cd438dba96f8cbe1863611" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 45317 } }
I20260808 22:59:50.505463 27101 raft_consensus.cc:388] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:50.505488 27101 raft_consensus.cc:482] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:50.505534 27101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:50.506021 27101 raft_consensus.cc:504] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865550ce39cd438dba96f8cbe1863611" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 45317 } }
I20260808 22:59:50.506107 27101 leader_election.cc:302] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 865550ce39cd438dba96f8cbe1863611; no voters:
I20260808 22:59:50.506268 27101 leader_election.cc:288] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:50.506304 27107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:50.506451 27107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611 [term 1 LEADER]: Becoming Leader. State: Replica: 865550ce39cd438dba96f8cbe1863611, State: Running, Role: LEADER
I20260808 22:59:50.506506 27101 sys_catalog.cc:564] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:50.506556 27107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865550ce39cd438dba96f8cbe1863611" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 45317 } }
I20260808 22:59:50.506896 27108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "865550ce39cd438dba96f8cbe1863611" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865550ce39cd438dba96f8cbe1863611" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 45317 } } }
I20260808 22:59:50.507015 27108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:50.506902 27109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 865550ce39cd438dba96f8cbe1863611. Latest consensus state: current_term: 1 leader_uuid: "865550ce39cd438dba96f8cbe1863611" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865550ce39cd438dba96f8cbe1863611" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 45317 } } }
I20260808 22:59:50.507164 27109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:50.507517 27118 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:50.507910 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 26988
I20260808 22:59:50.507999 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 22:59:50.508010 27118 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:50.508169 24328 external_mini_cluster.cc:1428] Setting key 65213e0e29aa1e16aaa1fe893558cb09
I20260808 22:59:50.509563 27118 catalog_manager.cc:1347] Generated new cluster ID: e3fef14ce3944d339dd63683840c1ee4
I20260808 22:59:50.509618 27118 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 22:59:50 dist-test-slave-f0tn krb5kdc[26961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:50.519860 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:50.508688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32828 (local address 127.23.194.62:45317)
0808 22:59:50.508848 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:50.508856 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:50.508886 (+ 30us) server_negotiation.cc:407] Connection header received
0808 22:59:50.508993 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:50.508999 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:50.509053 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:50.509193 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:50.509991 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.511313 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.512358 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.512964 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.513047 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:50.516060 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:50.516064 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:50.516064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:50.516075 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:50.518065 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:50.518460 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:50.518463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:50.518464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:50.518581 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:50.518811 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:50.518812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:50.518812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:50.519115 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:50.519236 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:50.519507 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 22:59:50.519662 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
I20260808 22:59:50.526814 27118 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:50.527307 27118 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:50.533152 27118 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611: Generated new TSK 0
I20260808 22:59:50.533325 27118 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
I20260808 22:59:50.557703 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:45317
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:34647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:50.636488 27139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:50.636682 27139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:50.636714 27139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:50.637841 27139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:50.637881 27139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:50.637908 27139 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 22:59:50.637923 27139 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:59:50.637945 27139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 22:59:50.639375 27139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:34647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.194.62:45317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:50.639596 27139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:50.639801 27139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:50.644497 27154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:50.644534 27150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:50.644534 27151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:50.644762 27139 server_base.cc:1034] running on GCE node
I20260808 22:59:50.644935 27139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:50.645172 27139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:50.646308 27139 hybrid_clock.cc:648] HybridClock initialized: now 1786229990646295 us; error 30 us; skew 500 ppm
Aug 08 22:59:50 dist-test-slave-f0tn krb5kdc[26961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229990, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:50.649255 27139 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 22:59:50.649712 27139 webserver.cc:460] Webserver started at http://127.23.194.1:44929/ using document root <none> and password file <none>
I20260808 22:59:50.649909 27139 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:50.649957 27139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:50.650081 27139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:50.651130 27139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "08e21cbce6254c83bcddee69082c86fe"
format_stamp: "Formatted at 2026-08-08 22:59:50 on dist-test-slave-f0tn"
server_key: "0ad8f9cda6dbecb65a1ce6db7fb21ac2"
server_key_iv: "e230eea92cb1a4ad1785c6d607510bea"
server_key_version: "encryptionkey@0"
I20260808 22:59:50.651481 27139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "08e21cbce6254c83bcddee69082c86fe"
format_stamp: "Formatted at 2026-08-08 22:59:50 on dist-test-slave-f0tn"
server_key: "0ad8f9cda6dbecb65a1ce6db7fb21ac2"
server_key_iv: "e230eea92cb1a4ad1785c6d607510bea"
server_key_version: "encryptionkey@0"
I20260808 22:59:50.652668 27139 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:59:50.653308 27163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:50.653463 27139 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 22:59:50.653546 27139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "08e21cbce6254c83bcddee69082c86fe"
format_stamp: "Formatted at 2026-08-08 22:59:50 on dist-test-slave-f0tn"
server_key: "0ad8f9cda6dbecb65a1ce6db7fb21ac2"
server_key_iv: "e230eea92cb1a4ad1785c6d607510bea"
server_key_version: "encryptionkey@0"
I20260808 22:59:50.653610 27139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:50.671989 27139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:50.673671 27139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:50.673800 27139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:50.674321 27139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:59:50.674361 27139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:50.674389 27139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:59:50.674422 27139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:59:50 dist-test-slave-f0tn krb5kdc[26961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229990, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:50.681118 27139 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:39133
I20260808 22:59:50.681489 27348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:39133 every 8 connection(s)
I20260808 22:59:50.681555 27139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 22:59:50.682255 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 27139
I20260808 22:59:50.682338 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 22:59:50.682507 24328 external_mini_cluster.cc:1428] Setting key 20f2d3e78cf1c69c7036ccf1559830e8
I20260808 22:59:50.683874 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:50.674700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:39427 (local address 127.23.194.62:45317)
0808 22:59:50.674765 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:50.674767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:50.674940 (+ 173us) server_negotiation.cc:407] Connection header received
0808 22:59:50.675393 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:50.675395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:50.675411 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:50.675492 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:50.676077 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.676830 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.677912 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.678114 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.678173 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:50.680948 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:50.680950 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:50.680950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:50.680957 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:50.682475 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:50.682891 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:50.682892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:50.682893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:50.682953 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:50.683205 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:50.683206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:50.683206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:50.683309 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:50.683371 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:50.683697 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 22:59:50.683798 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:59:50.684013 27176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:50.674666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:45317 (local address 127.23.194.1:39427)
0808 22:59:50.674815 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:50.674855 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:50.675302 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:50.675542 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:50.675545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:50.675778 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:50.675978 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:50.675983 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.676923 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.676925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:50.677830 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:50.677833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.678186 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.678187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:50.678310 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:50.678797 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:50.678806 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:50.680853 (+ 2047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:50.682543 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:50.682551 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:50.682553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:50.682826 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:50.683017 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:50.683017 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:50.683018 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:50.683146 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:50.683393 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:50.683399 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:50.683541 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 22:59:50.683656 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 22:59:50.683811 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":83,"threads_started":1}
I20260808 22:59:50.685549 27349 heartbeater.cc:347] Connected to a master server at 127.23.194.62:45317
I20260808 22:59:50.685627 27349 heartbeater.cc:464] Registering TS with master...
I20260808 22:59:50.685798 27349 heartbeater.cc:511] Master 127.23.194.62:45317 requested a full tablet report, sending...
I20260808 22:59:50.686319 27032 ts_manager.cc:194] Registered new tserver with Master: 08e21cbce6254c83bcddee69082c86fe (127.23.194.1:39133)
I20260808 22:59:50.686996 27032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:39427
WARNING: no policy specified for kudu/127.23.194.2@KRBTEST.COM; defaulting to no policy
I20260808 22:59:50.692085 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:50.686628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:40549 (local address 127.23.194.62:45317)
0808 22:59:50.686678 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:50.686680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:50.686790 (+ 110us) server_negotiation.cc:407] Connection header received
0808 22:59:50.686865 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:50.686866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:50.686875 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:50.686899 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:50.687349 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.688051 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.689038 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.689234 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.689290 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:50.690167 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:50.690167 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:50.690168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:50.690173 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:50.691325 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:50.691610 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:50.691611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:50.691611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:50.691663 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:50.691780 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:50.691781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:50.691781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:50.691861 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:50.691901 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:50.691979 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 22:59:50.692033 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 22:59:50.692113 27363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:50.686565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:45317 (local address 127.23.194.1:40549)
0808 22:59:50.686739 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:50.686748 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:50.686810 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:50.686951 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:50.686951 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:50.687201 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:50.687259 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:50.687261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.688166 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.688167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:50.688953 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:50.688956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.689304 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.689305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:50.689398 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:50.689737 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:50.689742 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:50.690080 (+ 338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:50.691411 (+ 1331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:50.691411 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:50.691412 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:50.691560 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:50.691708 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:50.691708 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:50.691708 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:50.691749 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:50.691894 (+ 145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:50.691894 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:50.691924 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 22:59:50.691965 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 22:59:50.692036 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":59,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
I20260808 22:59:50.705143 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:45317
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:34647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:50.784730 27367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:50.784921 27367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:50.784946 27367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:50.786115 27367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:50.786170 27367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:50.786209 27367 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 22:59:50.786238 27367 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:59:50.786273 27367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 22:59:50.787698 27367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:34647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.194.62:45317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:50.787909 27367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:50.788132 27367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:50.792826 27378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:50.792826 27377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:50.792959 27367 server_base.cc:1034] running on GCE node
W20260808 22:59:50.792840 27380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:50.793179 27367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:50.793409 27367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:50.794545 27367 hybrid_clock.cc:648] HybridClock initialized: now 1786229990794535 us; error 31 us; skew 500 ppm
Aug 08 22:59:50 dist-test-slave-f0tn krb5kdc[26961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229990, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:50.796790 27367 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 22:59:50.797153 27367 webserver.cc:460] Webserver started at http://127.23.194.2:46815/ using document root <none> and password file <none>
I20260808 22:59:50.797319 27367 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:50.797355 27367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:50.797487 27367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:50.798400 27367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "a955e828f15f4966a96f217dbecdb2ac"
format_stamp: "Formatted at 2026-08-08 22:59:50 on dist-test-slave-f0tn"
server_key: "8cadc2e8ba29adac204e261e435488a6"
server_key_iv: "7e2984e2adf90d87c273e68b52069bbf"
server_key_version: "encryptionkey@0"
I20260808 22:59:50.798691 27367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "a955e828f15f4966a96f217dbecdb2ac"
format_stamp: "Formatted at 2026-08-08 22:59:50 on dist-test-slave-f0tn"
server_key: "8cadc2e8ba29adac204e261e435488a6"
server_key_iv: "7e2984e2adf90d87c273e68b52069bbf"
server_key_version: "encryptionkey@0"
I20260808 22:59:50.799891 27367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 22:59:50.800508 27392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:50.800670 27367 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 22:59:50.800752 27367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "a955e828f15f4966a96f217dbecdb2ac"
format_stamp: "Formatted at 2026-08-08 22:59:50 on dist-test-slave-f0tn"
server_key: "8cadc2e8ba29adac204e261e435488a6"
server_key_iv: "7e2984e2adf90d87c273e68b52069bbf"
server_key_version: "encryptionkey@0"
I20260808 22:59:50.800803 27367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:50.817286 27367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:50.819399 27367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:50.819509 27367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:50.819972 27367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:59:50.820011 27367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:50.820032 27367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:59:50.820043 27367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:50.826339 27367 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:40651
I20260808 22:59:50.826398 27572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:40651 every 8 connection(s)
Aug 08 22:59:50 dist-test-slave-f0tn krb5kdc[26961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229990, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:50.826745 27367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 22:59:50.829245 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:50.820433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:59097 (local address 127.23.194.62:45317)
0808 22:59:50.820500 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:50.820502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:50.820660 (+ 158us) server_negotiation.cc:407] Connection header received
0808 22:59:50.821142 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:50.821143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:50.821157 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:50.821229 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:50.822064 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.822810 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.824264 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.824555 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.824624 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:50.827108 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:50.827109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:50.827110 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:50.827118 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:50.828206 (+ 1088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:50.828573 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:50.828574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:50.828574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:50.828617 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:50.828794 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:50.828795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:50.828795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:50.828884 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:50.828927 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:50.829126 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 22:59:50.829184 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 22:59:50.829355 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 27367
I20260808 22:59:50.829478 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 22:59:50.829551 27406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:50.820380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:45317 (local address 127.23.194.2:59097)
0808 22:59:50.820535 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:50.820576 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:50.821054 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:50.821294 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:50.821297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:50.821668 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:50.821959 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:50.821965 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.822908 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.822909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:50.824168 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:50.824172 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.824636 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.824637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:50.824744 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:50.825241 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:50.825251 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:50.827018 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:50.828270 (+ 1252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:50.828276 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:50.828277 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:50.828518 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:50.828667 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:50.828668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:50.828668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:50.828756 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:50.828925 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:50.828928 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:50.829033 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 22:59:50.829108 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 22:59:50.829252 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260808 22:59:50.829687 24328 external_mini_cluster.cc:1428] Setting key a687e8c2900387860a640c34697ea28c
I20260808 22:59:50.830021 27575 heartbeater.cc:347] Connected to a master server at 127.23.194.62:45317
I20260808 22:59:50.830103 27575 heartbeater.cc:464] Registering TS with master...
I20260808 22:59:50.830286 27575 heartbeater.cc:511] Master 127.23.194.62:45317 requested a full tablet report, sending...
I20260808 22:59:50.830601 27033 ts_manager.cc:194] Registered new tserver with Master: a955e828f15f4966a96f217dbecdb2ac (127.23.194.2:40651)
I20260808 22:59:50.830997 27033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:59097
I20260808 22:59:50.836192 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:50.831087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:58097 (local address 127.23.194.62:45317)
0808 22:59:50.831172 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:50.831174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:50.831457 (+ 283us) server_negotiation.cc:407] Connection header received
0808 22:59:50.831511 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:50.831512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:50.831521 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:50.831564 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:50.831847 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.832299 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.832943 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.833132 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.833185 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:50.834009 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:50.834011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:50.834011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:50.834017 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:50.835209 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:50.835494 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:50.835495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:50.835495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:50.835545 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:50.835701 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:50.835702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:50.835702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:50.835837 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:50.835893 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:50.836037 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 22:59:50.836106 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 22:59:50.836213 27584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:50.831026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:45317 (local address 127.23.194.2:58097)
0808 22:59:50.831393 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:50.831403 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:50.831459 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:50.831567 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:50.831568 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:50.831731 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:50.831784 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:50.831785 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.832390 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.832391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:50.832877 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:50.832878 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.833190 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.833191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:50.833262 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:50.833636 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:50.833641 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:50.833933 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:50.835278 (+ 1345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:50.835279 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:50.835279 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:50.835437 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:50.835596 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:50.835597 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:50.835597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:50.835652 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:50.835895 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:50.835896 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:50.835940 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 22:59:50.836009 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 22:59:50.836108 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":303,"threads_started":1}
WARNING: no policy specified for kudu/127.23.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
I20260808 22:59:50.853209 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:45317
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:34647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:50.933322 27588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:50.933526 27588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:50.933549 27588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:50.934646 27588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:50.934684 27588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:50.934706 27588 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 22:59:50.934729 27588 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:59:50.934751 27588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 22:59:50.936193 27588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:34647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.194.62:45317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:50.936401 27588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:50.936596 27588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:50.941246 27603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:50.941250 27597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:50.941290 27600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:50.941512 27588 server_base.cc:1034] running on GCE node
I20260808 22:59:50.941689 27588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:50.941918 27588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:50.943055 27588 hybrid_clock.cc:648] HybridClock initialized: now 1786229990943041 us; error 27 us; skew 500 ppm
Aug 08 22:59:50 dist-test-slave-f0tn krb5kdc[26961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229990, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:50.945281 27588 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 22:59:50.945729 27588 webserver.cc:460] Webserver started at http://127.23.194.3:44507/ using document root <none> and password file <none>
I20260808 22:59:50.945941 27588 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:50.945988 27588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:50.946110 27588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:50.947057 27588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "dafd6b7d77a444b6aaa6deb58300d9eb"
format_stamp: "Formatted at 2026-08-08 22:59:50 on dist-test-slave-f0tn"
server_key: "d6efaf4e4a72d5361156dfd3b45b1a3a"
server_key_iv: "3b7b688fc54985e23d198b47808bbacd"
server_key_version: "encryptionkey@0"
I20260808 22:59:50.947388 27588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "dafd6b7d77a444b6aaa6deb58300d9eb"
format_stamp: "Formatted at 2026-08-08 22:59:50 on dist-test-slave-f0tn"
server_key: "d6efaf4e4a72d5361156dfd3b45b1a3a"
server_key_iv: "3b7b688fc54985e23d198b47808bbacd"
server_key_version: "encryptionkey@0"
I20260808 22:59:50.948547 27588 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 22:59:50.949170 27614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:50.949321 27588 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 22:59:50.949411 27588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "dafd6b7d77a444b6aaa6deb58300d9eb"
format_stamp: "Formatted at 2026-08-08 22:59:50 on dist-test-slave-f0tn"
server_key: "d6efaf4e4a72d5361156dfd3b45b1a3a"
server_key_iv: "3b7b688fc54985e23d198b47808bbacd"
server_key_version: "encryptionkey@0"
I20260808 22:59:50.949473 27588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:50.972882 27588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:50.974650 27588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:50.974790 27588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:50.975289 27588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:59:50.975339 27588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:50.975371 27588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:59:50.975399 27588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:59:50 dist-test-slave-f0tn krb5kdc[26961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229990, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:50.982865 27588 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:40031
I20260808 22:59:50.982930 27794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:40031 every 8 connection(s)
I20260808 22:59:50.983237 27588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 22:59:50.984929 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:50.975738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:43547 (local address 127.23.194.62:45317)
0808 22:59:50.975810 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:50.975813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:50.975964 (+ 151us) server_negotiation.cc:407] Connection header received
0808 22:59:50.976421 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:50.976422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:50.976436 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:50.976527 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:50.977092 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.977847 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.978860 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.979152 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.979233 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:50.982130 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:50.982131 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:50.982131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:50.982139 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:50.983609 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:50.984042 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:50.984044 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:50.984044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:50.984103 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:50.984340 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:50.984341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:50.984341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:50.984453 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:50.984508 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:50.984764 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 22:59:50.984843 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:59:50.985081 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:50.975694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:45317 (local address 127.23.194.3:43547)
0808 22:59:50.975844 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:50.975883 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:50.976332 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:50.976573 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:50.976576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:50.976786 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:50.977013 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:50.977018 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.977955 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.977956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:50.978772 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:50.978774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.979240 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.979241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:50.979360 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:50.979832 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:50.979843 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:50.982027 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:50.983701 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:50.983705 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:50.983707 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:50.983944 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:50.984171 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:50.984172 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:50.984173 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:50.984281 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:50.984509 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:50.984515 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:50.984642 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 22:59:50.984731 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 22:59:50.984881 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260808 22:59:50.985576 27796 heartbeater.cc:347] Connected to a master server at 127.23.194.62:45317
I20260808 22:59:50.985670 27796 heartbeater.cc:464] Registering TS with master...
I20260808 22:59:50.985843 27796 heartbeater.cc:511] Master 127.23.194.62:45317 requested a full tablet report, sending...
I20260808 22:59:50.986249 27032 ts_manager.cc:194] Registered new tserver with Master: dafd6b7d77a444b6aaa6deb58300d9eb (127.23.194.3:40031)
I20260808 22:59:50.986742 27032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:43547
I20260808 22:59:50.987887 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 27588
I20260808 22:59:50.987983 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 22:59:50.988173 24328 external_mini_cluster.cc:1428] Setting key fcc585646058ff1c3b7cf5f99e713010
I20260808 22:59:50.989048 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:59:50.991676 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:50.986543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:45317 (local address 127.23.194.3:49405)
0808 22:59:50.986686 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:50.986696 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:50.986749 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:50.986941 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:50.986942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:50.987085 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:50.987139 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:50.987141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.987765 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.987766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:50.988346 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:50.988348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.988773 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.988774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:50.988848 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:50.989243 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:50.989247 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:50.989537 (+ 290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:50.990783 (+ 1246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:50.990783 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:50.990784 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:50.990937 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:50.991103 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:50.991103 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:50.991104 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:50.991159 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:50.991407 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:50.991408 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:50.991452 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 22:59:50.991525 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 22:59:50.991605 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
I20260808 22:59:50.991703 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:50.986558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:49405 (local address 127.23.194.62:45317)
0808 22:59:50.986600 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:50.986602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:50.986749 (+ 147us) server_negotiation.cc:407] Connection header received
0808 22:59:50.986802 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:50.986803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:50.986812 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:50.986854 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:50.987211 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.987664 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.988435 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:50.988699 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:50.988769 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:50.989610 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:50.989612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:50.989612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:50.989618 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:50.990714 (+ 1096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:50.990995 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:50.990996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:50.990996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:50.991046 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:50.991208 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:50.991209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:50.991209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:50.991329 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:50.991392 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:50.991555 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 22:59:50.991626 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
Aug 08 22:59:50 dist-test-slave-f0tn krb5kdc[26961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 22:59:51 dist-test-slave-f0tn krb5kdc[26961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:51.012390 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.006027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32830 (local address 127.23.194.62:45317)
0808 22:59:51.006078 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:51.006080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:51.006090 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:59:51.006114 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:51.006115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:51.006124 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:51.006167 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:51.006750 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.007280 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.008040 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.008225 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.008268 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.010354 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:51.010356 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:51.010356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:51.010362 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:51.011568 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:51.011848 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:51.011849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:51.011849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:51.011900 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:51.012023 (+ 123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:51.012024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:51.012024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:51.012127 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:51.012203 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:51.012268 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 22:59:51.012327 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 22:59:51.013870 27032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:59:51.015312 27032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:59:51.021492 27816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.017309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:39133 (local address 127.0.0.1:38196)
0808 22:59:51.017624 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:51.017665 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:51.017729 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:51.018007 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:51.018008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:51.018010 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:51.018083 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.018085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.019403 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.019404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.020442 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.020445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.021160 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.021161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.021242 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.021258 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 22:59:51.021324 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 22:59:51.021411 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":24,"thread_start_us":58,"threads_started":1}
I20260808 22:59:51.021524 27825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.017674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38196 (local address 127.23.194.1:39133)
0808 22:59:51.017836 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:51.017841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:51.017853 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:59:51.017896 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:51.017898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:51.017915 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:51.018012 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:51.018182 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.019309 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.020530 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.021076 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.021151 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.021218 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:51.021348 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 22:59:51.021413 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260808 22:59:51.021641 27822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.017448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48214 (local address 127.23.194.3:40031)
0808 22:59:51.017662 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:51.017667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:51.017805 (+ 138us) server_negotiation.cc:407] Connection header received
0808 22:59:51.017843 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:51.017845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:51.017859 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:51.017921 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:51.018134 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.018935 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.020422 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.021120 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.021196 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.021248 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:51.021529 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 22:59:51.021567 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":108,"threads_started":1}
I20260808 22:59:51.021744 27820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.017514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:40651 (local address 127.0.0.1:58374)
0808 22:59:51.017849 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:51.017860 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:51.017918 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:51.018396 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:51.018396 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:51.018398 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:51.018433 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.018435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.019593 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.019595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.020537 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.020540 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.021521 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.021522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.021577 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.021580 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:59:51.021625 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 22:59:51.021671 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":46,"threads_started":1}
I20260808 22:59:51.021939 27823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.017309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:40031 (local address 127.0.0.1:48214)
0808 22:59:51.017706 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:51.017715 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:51.017758 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:51.018002 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:51.018005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:51.018008 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:51.018076 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.018078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.019002 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.019003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.020151 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.020153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.021283 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.021285 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.021345 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.021349 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:59:51.021424 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 22:59:51.021465 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":2,"thread_start_us":47,"threads_started":1}
I20260808 22:59:51.022037 27824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.017633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58374 (local address 127.23.194.2:40651)
0808 22:59:51.018233 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:51.018237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:51.018247 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:59:51.018287 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:51.018288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:51.018303 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:51.018388 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:51.018511 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.019517 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.020634 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.021333 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.021432 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.021487 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:51.021901 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 22:59:51.021941 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":90,"threads_started":1}
I20260808 22:59:51.022465 27244 tablet_service.cc:1467] Processing CreateTablet for tablet c8b5f5c9a57742398bf9bbb1abdff58c (DEFAULT_TABLE table=test-table [id=47251987dca543d3874ca9cab741d568]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:51.022544 27694 tablet_service.cc:1467] Processing CreateTablet for tablet c8b5f5c9a57742398bf9bbb1abdff58c (DEFAULT_TABLE table=test-table [id=47251987dca543d3874ca9cab741d568]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:51.022572 27466 tablet_service.cc:1467] Processing CreateTablet for tablet c8b5f5c9a57742398bf9bbb1abdff58c (DEFAULT_TABLE table=test-table [id=47251987dca543d3874ca9cab741d568]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:51.022861 27244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8b5f5c9a57742398bf9bbb1abdff58c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:51.022861 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8b5f5c9a57742398bf9bbb1abdff58c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:51.022888 27466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8b5f5c9a57742398bf9bbb1abdff58c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:51.025503 27827 tablet_bootstrap.cc:492] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac: Bootstrap starting.
I20260808 22:59:51.026134 27827 tablet_bootstrap.cc:654] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:51.026141 27828 tablet_bootstrap.cc:492] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb: Bootstrap starting.
I20260808 22:59:51.026396 27829 tablet_bootstrap.cc:492] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe: Bootstrap starting.
I20260808 22:59:51.026779 27827 log.cc:824] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:51.026953 27828 tablet_bootstrap.cc:654] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:51.027217 27828 log.cc:824] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:51.027316 27829 tablet_bootstrap.cc:654] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:51.027618 27827 tablet_bootstrap.cc:492] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac: No bootstrap required, opened a new log
I20260808 22:59:51.027606 27829 log.cc:824] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:51.027678 27827 ts_tablet_manager.cc:1397] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:59:51.028380 27829 tablet_bootstrap.cc:492] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe: No bootstrap required, opened a new log
I20260808 22:59:51.028435 27829 ts_tablet_manager.cc:1397] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:59:51.029116 27827 raft_consensus.cc:348] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } }
I20260808 22:59:51.029269 27827 raft_consensus.cc:374] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:51.029301 27827 raft_consensus.cc:729] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a955e828f15f4966a96f217dbecdb2ac, State: Initialized, Role: FOLLOWER
I20260808 22:59:51.029408 27827 consensus_queue.cc:260] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } }
I20260808 22:59:51.029678 27827 ts_tablet_manager.cc:1428] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:59:51.029735 27575 heartbeater.cc:503] Master 127.23.194.62:45317 was elected leader, sending a full tablet report...
I20260808 22:59:51.030099 27829 raft_consensus.cc:348] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } }
I20260808 22:59:51.030268 27829 raft_consensus.cc:374] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:51.030311 27828 tablet_bootstrap.cc:492] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb: No bootstrap required, opened a new log
I20260808 22:59:51.030314 27829 raft_consensus.cc:729] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e21cbce6254c83bcddee69082c86fe, State: Initialized, Role: FOLLOWER
I20260808 22:59:51.030364 27828 ts_tablet_manager.cc:1397] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 22:59:51.030422 27829 consensus_queue.cc:260] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } }
I20260808 22:59:51.030680 27829 ts_tablet_manager.cc:1428] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 22:59:51.031806 27828 raft_consensus.cc:348] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } }
I20260808 22:59:51.031921 27828 raft_consensus.cc:374] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:51.031940 27828 raft_consensus.cc:729] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dafd6b7d77a444b6aaa6deb58300d9eb, State: Initialized, Role: FOLLOWER
I20260808 22:59:51.032021 27828 consensus_queue.cc:260] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } }
I20260808 22:59:51.032241 27828 ts_tablet_manager.cc:1428] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:59:51.032297 27796 heartbeater.cc:503] Master 127.23.194.62:45317 was elected leader, sending a full tablet report...
I20260808 22:59:51.032367 27349 heartbeater.cc:503] Master 127.23.194.62:45317 was elected leader, sending a full tablet report...
I20260808 22:59:51.044075 27833 raft_consensus.cc:482] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:59:51.044154 27833 raft_consensus.cc:504] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } }
I20260808 22:59:51.044401 27833 leader_election.cc:288] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08e21cbce6254c83bcddee69082c86fe (127.23.194.1:39133), dafd6b7d77a444b6aaa6deb58300d9eb (127.23.194.3:40031)
I20260808 22:59:51.047683 27825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.044612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:37669 (local address 127.23.194.1:39133)
0808 22:59:51.044658 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:51.044660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:51.044891 (+ 231us) server_negotiation.cc:407] Connection header received
0808 22:59:51.044915 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:51.044915 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:51.044924 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:51.044976 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:51.045097 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.045621 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.046830 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.047363 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.047426 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.047435 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:51.047583 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 22:59:51.047615 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 22:59:51.047680 27822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.044619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:50847 (local address 127.23.194.3:40031)
0808 22:59:51.044665 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:51.044666 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:51.044675 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:59:51.044713 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:51.044713 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:51.044720 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:51.044763 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:51.045010 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.045517 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.046820 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.047355 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.047426 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.047434 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:51.047583 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 22:59:51.047615 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 22:59:51.047710 27406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.044550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:40031 (local address 127.23.194.2:50847)
0808 22:59:51.044588 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:51.044599 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:51.044652 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:51.044780 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:51.044781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:51.044784 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:51.044843 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.044844 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.045591 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.045592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.046736 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.046738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.047431 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.047432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.047517 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.047519 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 22:59:51.047573 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 22:59:51.047616 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":7}
I20260808 22:59:51.047879 27837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.044551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:39133 (local address 127.23.194.2:37669)
0808 22:59:51.044825 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:51.044834 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:51.044880 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:51.044980 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:51.044980 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:51.044982 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:51.045036 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.045038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.045700 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.045701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.046758 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.046760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.047431 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.047432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.047513 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.047517 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:59:51.047573 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 22:59:51.047616 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":147,"thread_start_us":80,"threads_started":1}
I20260808 22:59:51.048069 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8b5f5c9a57742398bf9bbb1abdff58c" candidate_uuid: "a955e828f15f4966a96f217dbecdb2ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" is_pre_election: true
I20260808 22:59:51.048198 27721 raft_consensus.cc:2454] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a955e828f15f4966a96f217dbecdb2ac in term 0.
I20260808 22:59:51.048187 27281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8b5f5c9a57742398bf9bbb1abdff58c" candidate_uuid: "a955e828f15f4966a96f217dbecdb2ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08e21cbce6254c83bcddee69082c86fe" is_pre_election: true
I20260808 22:59:51.048317 27281 raft_consensus.cc:2454] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a955e828f15f4966a96f217dbecdb2ac in term 0.
I20260808 22:59:51.048406 27398 leader_election.cc:302] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a955e828f15f4966a96f217dbecdb2ac, dafd6b7d77a444b6aaa6deb58300d9eb; no voters:
I20260808 22:59:51.048534 27833 raft_consensus.cc:2781] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:59:51.048590 27833 raft_consensus.cc:482] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:59:51.048617 27833 raft_consensus.cc:3037] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:51.049258 27833 raft_consensus.cc:504] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } }
I20260808 22:59:51.049371 27833 leader_election.cc:288] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [CANDIDATE]: Term 1 election: Requested vote from peers 08e21cbce6254c83bcddee69082c86fe (127.23.194.1:39133), dafd6b7d77a444b6aaa6deb58300d9eb (127.23.194.3:40031)
I20260808 22:59:51.049536 27281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8b5f5c9a57742398bf9bbb1abdff58c" candidate_uuid: "a955e828f15f4966a96f217dbecdb2ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08e21cbce6254c83bcddee69082c86fe"
I20260808 22:59:51.049553 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8b5f5c9a57742398bf9bbb1abdff58c" candidate_uuid: "a955e828f15f4966a96f217dbecdb2ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb"
I20260808 22:59:51.049608 27281 raft_consensus.cc:3037] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:51.049613 27721 raft_consensus.cc:3037] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:51.050215 27281 raft_consensus.cc:2454] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a955e828f15f4966a96f217dbecdb2ac in term 1.
I20260808 22:59:51.050348 27721 raft_consensus.cc:2454] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a955e828f15f4966a96f217dbecdb2ac in term 1.
I20260808 22:59:51.050801 27396 leader_election.cc:302] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08e21cbce6254c83bcddee69082c86fe, a955e828f15f4966a96f217dbecdb2ac; no voters:
I20260808 22:59:51.050923 27833 raft_consensus.cc:2781] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:51.051062 27833 raft_consensus.cc:686] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [term 1 LEADER]: Becoming Leader. State: Replica: a955e828f15f4966a96f217dbecdb2ac, State: Running, Role: LEADER
I20260808 22:59:51.051146 27833 consensus_queue.cc:237] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } }
I20260808 22:59:51.051920 27032 catalog_manager.cc:5495] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac reported cstate change: term changed from 0 to 1, leader changed from <none> to a955e828f15f4966a96f217dbecdb2ac (127.23.194.2). New cstate: current_term: 1 leader_uuid: "a955e828f15f4966a96f217dbecdb2ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } health_report { overall_health: HEALTHY } } }
W20260808 22:59:51.077268 27576 tablet.cc:2369] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 22:59:51 dist-test-slave-f0tn krb5kdc[26961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229990, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:51.100544 27849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.094123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:45317 (local address 127.0.0.1:32834)
0808 22:59:51.094293 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:51.094304 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:51.094517 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:51.094646 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:51.094646 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:51.094962 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:51.095014 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.095016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.095605 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.095605 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.096158 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.096159 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.096426 (+ 267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.096427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.096463 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.096979 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:51.096984 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:51.098399 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:51.099675 (+ 1276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:51.099683 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:51.099685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:51.099876 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:51.100023 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:51.100024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:51.100024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:51.100074 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:51.100268 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:51.100272 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:51.100310 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 22:59:51.100356 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 22:59:51.100450 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":1,"thread_start_us":75,"threads_started":1}
I20260808 22:59:51.100594 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.094200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32834 (local address 127.23.194.62:45317)
0808 22:59:51.094279 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:51.094281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:51.094503 (+ 222us) server_negotiation.cc:407] Connection header received
0808 22:59:51.094570 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:51.094571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:51.094580 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:51.094630 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:51.095072 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.095550 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.096212 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.096380 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.096422 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.098477 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:51.098479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:51.098479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:51.098486 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:51.099598 (+ 1112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:51.099934 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:51.099934 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:51.099935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:51.099980 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:51.100131 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:51.100132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:51.100132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:51.100227 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:51.100272 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:51.100377 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 22:59:51.100495 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":46}
I20260808 22:59:51.102608 27032 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:32834:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 22:59:51.102830 27032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:59:51.104327 27244 tablet_service.cc:1467] Processing CreateTablet for tablet a347d157cd6944c88275313d8094bd5c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=16590ae8193c46af98a8909942d1221f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:59:51.104341 27694 tablet_service.cc:1467] Processing CreateTablet for tablet a347d157cd6944c88275313d8094bd5c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=16590ae8193c46af98a8909942d1221f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:59:51.104341 27466 tablet_service.cc:1467] Processing CreateTablet for tablet a347d157cd6944c88275313d8094bd5c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=16590ae8193c46af98a8909942d1221f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:59:51.104496 27466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a347d157cd6944c88275313d8094bd5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:51.104496 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a347d157cd6944c88275313d8094bd5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:51.104494 27244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a347d157cd6944c88275313d8094bd5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:51.105758 27829 tablet_bootstrap.cc:492] T a347d157cd6944c88275313d8094bd5c P 08e21cbce6254c83bcddee69082c86fe: Bootstrap starting.
I20260808 22:59:51.106014 27827 tablet_bootstrap.cc:492] T a347d157cd6944c88275313d8094bd5c P a955e828f15f4966a96f217dbecdb2ac: Bootstrap starting.
I20260808 22:59:51.106043 27828 tablet_bootstrap.cc:492] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb: Bootstrap starting.
I20260808 22:59:51.106451 27827 tablet_bootstrap.cc:654] T a347d157cd6944c88275313d8094bd5c P a955e828f15f4966a96f217dbecdb2ac: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:51.106451 27829 tablet_bootstrap.cc:654] T a347d157cd6944c88275313d8094bd5c P 08e21cbce6254c83bcddee69082c86fe: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:51.106503 27828 tablet_bootstrap.cc:654] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:51.107172 27829 tablet_bootstrap.cc:492] T a347d157cd6944c88275313d8094bd5c P 08e21cbce6254c83bcddee69082c86fe: No bootstrap required, opened a new log
I20260808 22:59:51.107172 27827 tablet_bootstrap.cc:492] T a347d157cd6944c88275313d8094bd5c P a955e828f15f4966a96f217dbecdb2ac: No bootstrap required, opened a new log
I20260808 22:59:51.107224 27829 ts_tablet_manager.cc:1397] T a347d157cd6944c88275313d8094bd5c P 08e21cbce6254c83bcddee69082c86fe: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:59:51.107224 27827 ts_tablet_manager.cc:1397] T a347d157cd6944c88275313d8094bd5c P a955e828f15f4966a96f217dbecdb2ac: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:51.107372 27827 raft_consensus.cc:348] T a347d157cd6944c88275313d8094bd5c P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:51.107372 27829 raft_consensus.cc:348] T a347d157cd6944c88275313d8094bd5c P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:51.107456 27827 raft_consensus.cc:374] T a347d157cd6944c88275313d8094bd5c P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:51.107456 27829 raft_consensus.cc:374] T a347d157cd6944c88275313d8094bd5c P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:51.107477 27829 raft_consensus.cc:729] T a347d157cd6944c88275313d8094bd5c P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e21cbce6254c83bcddee69082c86fe, State: Initialized, Role: FOLLOWER
I20260808 22:59:51.107477 27827 raft_consensus.cc:729] T a347d157cd6944c88275313d8094bd5c P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a955e828f15f4966a96f217dbecdb2ac, State: Initialized, Role: FOLLOWER
I20260808 22:59:51.107527 27827 consensus_queue.cc:260] T a347d157cd6944c88275313d8094bd5c P a955e828f15f4966a96f217dbecdb2ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:51.107527 27829 consensus_queue.cc:260] T a347d157cd6944c88275313d8094bd5c P 08e21cbce6254c83bcddee69082c86fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:51.107625 27829 ts_tablet_manager.cc:1428] T a347d157cd6944c88275313d8094bd5c P 08e21cbce6254c83bcddee69082c86fe: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 22:59:51.107628 27827 ts_tablet_manager.cc:1428] T a347d157cd6944c88275313d8094bd5c P a955e828f15f4966a96f217dbecdb2ac: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:51.107729 27835 tablet_replica.cc:439] T a347d157cd6944c88275313d8094bd5c P 08e21cbce6254c83bcddee69082c86fe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } } }
I20260808 22:59:51.107821 27835 tablet_replica.cc:442] T a347d157cd6944c88275313d8094bd5c P 08e21cbce6254c83bcddee69082c86fe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:59:51.107959 27833 tablet_replica.cc:439] T a347d157cd6944c88275313d8094bd5c P a955e828f15f4966a96f217dbecdb2ac: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } } }
I20260808 22:59:51.108047 27833 tablet_replica.cc:442] T a347d157cd6944c88275313d8094bd5c P a955e828f15f4966a96f217dbecdb2ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:59:51.108140 27828 tablet_bootstrap.cc:492] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb: No bootstrap required, opened a new log
I20260808 22:59:51.108186 27828 ts_tablet_manager.cc:1397] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:59:51.108325 27828 raft_consensus.cc:348] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:51.108388 27828 raft_consensus.cc:374] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:51.108407 27828 raft_consensus.cc:729] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dafd6b7d77a444b6aaa6deb58300d9eb, State: Initialized, Role: FOLLOWER
I20260808 22:59:51.108446 27828 consensus_queue.cc:260] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:51.108527 27828 ts_tablet_manager.cc:1428] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:51.108588 27836 tablet_replica.cc:439] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } } }
I20260808 22:59:51.108704 27836 tablet_replica.cc:442] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 22:59:51.182220 27350 tablet.cc:2369] T a347d157cd6944c88275313d8094bd5c P 08e21cbce6254c83bcddee69082c86fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:59:51.208007 27836 raft_consensus.cc:482] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:59:51.208114 27836 raft_consensus.cc:504] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:51.208436 27836 leader_election.cc:288] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08e21cbce6254c83bcddee69082c86fe (127.23.194.1:39133), a955e828f15f4966a96f217dbecdb2ac (127.23.194.2:40651)
I20260808 22:59:51.211112 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.208607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:39133 (local address 127.23.194.3:50235)
0808 22:59:51.208665 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:51.208678 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:51.208723 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:51.208866 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:51.208868 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:51.208870 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:51.208942 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.208944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.209618 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.209619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.210349 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.210351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.210866 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.210867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.210946 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.210950 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:59:51.210995 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 22:59:51.211036 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 22:59:51.211112 27825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.208676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:50235 (local address 127.23.194.1:39133)
0808 22:59:51.208727 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:51.208729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:51.208770 (+ 41us) server_negotiation.cc:407] Connection header received
0808 22:59:51.208798 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:51.208799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:51.208809 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:51.208859 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:51.209003 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.209546 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.210436 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.210795 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.210854 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.210863 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:51.211008 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 22:59:51.211044 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 22:59:51.211295 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.208749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:40651 (local address 127.23.194.3:50317)
0808 22:59:51.208962 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:51.208970 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:51.209017 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:51.209127 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:51.209127 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:51.209129 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:51.209208 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.209210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.209817 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.209818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.210539 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.210541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.211062 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.211064 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.211130 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.211133 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:59:51.211179 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 22:59:51.211216 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":129,"threads_started":1}
I20260808 22:59:51.211289 27824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.208795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:50317 (local address 127.23.194.2:40651)
0808 22:59:51.208862 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:51.208864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:51.209027 (+ 163us) server_negotiation.cc:407] Connection header received
0808 22:59:51.209059 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:51.209060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:51.209069 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:51.209116 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:51.209269 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.209744 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.210626 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.210974 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.211033 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.211041 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:51.211195 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 22:59:51.211230 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:59:51.211616 27281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a347d157cd6944c88275313d8094bd5c" candidate_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08e21cbce6254c83bcddee69082c86fe" is_pre_election: true
I20260808 22:59:51.211720 27281 raft_consensus.cc:2454] T a347d157cd6944c88275313d8094bd5c P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dafd6b7d77a444b6aaa6deb58300d9eb in term 0.
I20260808 22:59:51.211711 27501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a347d157cd6944c88275313d8094bd5c" candidate_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a955e828f15f4966a96f217dbecdb2ac" is_pre_election: true
I20260808 22:59:51.211840 27501 raft_consensus.cc:2454] T a347d157cd6944c88275313d8094bd5c P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dafd6b7d77a444b6aaa6deb58300d9eb in term 0.
I20260808 22:59:51.211884 27616 leader_election.cc:302] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08e21cbce6254c83bcddee69082c86fe, dafd6b7d77a444b6aaa6deb58300d9eb; no voters:
I20260808 22:59:51.212007 27836 raft_consensus.cc:2781] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:59:51.212039 27836 raft_consensus.cc:482] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:59:51.212062 27836 raft_consensus.cc:3037] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:51.212742 27836 raft_consensus.cc:504] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:51.212860 27836 leader_election.cc:288] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb [CANDIDATE]: Term 1 election: Requested vote from peers 08e21cbce6254c83bcddee69082c86fe (127.23.194.1:39133), a955e828f15f4966a96f217dbecdb2ac (127.23.194.2:40651)
I20260808 22:59:51.213032 27501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a347d157cd6944c88275313d8094bd5c" candidate_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a955e828f15f4966a96f217dbecdb2ac"
I20260808 22:59:51.213043 27281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a347d157cd6944c88275313d8094bd5c" candidate_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08e21cbce6254c83bcddee69082c86fe"
I20260808 22:59:51.213104 27281 raft_consensus.cc:3037] T a347d157cd6944c88275313d8094bd5c P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:51.213104 27501 raft_consensus.cc:3037] T a347d157cd6944c88275313d8094bd5c P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:51.213740 27281 raft_consensus.cc:2454] T a347d157cd6944c88275313d8094bd5c P 08e21cbce6254c83bcddee69082c86fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dafd6b7d77a444b6aaa6deb58300d9eb in term 1.
I20260808 22:59:51.213828 27501 raft_consensus.cc:2454] T a347d157cd6944c88275313d8094bd5c P a955e828f15f4966a96f217dbecdb2ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dafd6b7d77a444b6aaa6deb58300d9eb in term 1.
I20260808 22:59:51.213901 27616 leader_election.cc:302] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08e21cbce6254c83bcddee69082c86fe, dafd6b7d77a444b6aaa6deb58300d9eb; no voters:
I20260808 22:59:51.213995 27836 raft_consensus.cc:2781] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:51.214126 27836 raft_consensus.cc:686] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb [term 1 LEADER]: Becoming Leader. State: Replica: dafd6b7d77a444b6aaa6deb58300d9eb, State: Running, Role: LEADER
I20260808 22:59:51.214213 27836 consensus_queue.cc:237] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:51.214633 27856 tablet_replica.cc:439] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb: TxnStatusTablet state changed. Reason: New leader dafd6b7d77a444b6aaa6deb58300d9eb. Latest consensus state: current_term: 1 leader_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } } }
I20260808 22:59:51.214721 27856 tablet_replica.cc:442] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:59:51.214974 27858 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 22:59:51.215090 27858 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 22:59:51.215034 27033 catalog_manager.cc:5495] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb reported cstate change: term changed from 0 to 1, leader changed from <none> to dafd6b7d77a444b6aaa6deb58300d9eb (127.23.194.3). New cstate: current_term: 1 leader_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } health_report { overall_health: HEALTHY } } }
I20260808 22:59:51.225196 27849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.222766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:40031 (local address 127.0.0.1:48224)
0808 22:59:51.222868 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:51.222882 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:51.222950 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:51.223121 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:51.223121 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:51.223122 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:59:51.223173 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.223175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.223793 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.223794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.224372 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.224373 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.224710 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.224711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.224780 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.224795 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:59:51.225027 (+ 232us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:59:51.225036 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 22:59:51.225077 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 22:59:51.225127 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 22:59:51.225185 27822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.222844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48224 (local address 127.23.194.3:40031)
0808 22:59:51.222889 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:51.222890 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:51.222928 (+ 38us) server_negotiation.cc:407] Connection header received
0808 22:59:51.223036 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:51.223037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:51.223055 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:51.223109 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:51.223240 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.223717 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.224452 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.224649 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.224703 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.224876 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:51.224972 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:51.225048 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:51.225090 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 22:59:51.225112 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 22:59:51.227115 27281 raft_consensus.cc:1261] T a347d157cd6944c88275313d8094bd5c P 08e21cbce6254c83bcddee69082c86fe [term 1 FOLLOWER]: Refusing update from remote peer dafd6b7d77a444b6aaa6deb58300d9eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:51.227160 27501 raft_consensus.cc:1261] T a347d157cd6944c88275313d8094bd5c P a955e828f15f4966a96f217dbecdb2ac [term 1 FOLLOWER]: Refusing update from remote peer dafd6b7d77a444b6aaa6deb58300d9eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:51.227425 27836 consensus_queue.cc:1035] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:51.227538 27856 consensus_queue.cc:1035] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:51.228698 27835 tablet_replica.cc:439] T a347d157cd6944c88275313d8094bd5c P 08e21cbce6254c83bcddee69082c86fe: TxnStatusTablet state changed. Reason: New leader dafd6b7d77a444b6aaa6deb58300d9eb. Latest consensus state: current_term: 1 leader_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } } }
I20260808 22:59:51.228727 27833 tablet_replica.cc:439] T a347d157cd6944c88275313d8094bd5c P a955e828f15f4966a96f217dbecdb2ac: TxnStatusTablet state changed. Reason: New leader dafd6b7d77a444b6aaa6deb58300d9eb. Latest consensus state: current_term: 1 leader_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } } }
I20260808 22:59:51.228785 27835 tablet_replica.cc:442] T a347d157cd6944c88275313d8094bd5c P 08e21cbce6254c83bcddee69082c86fe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:59:51.228804 27833 tablet_replica.cc:442] T a347d157cd6944c88275313d8094bd5c P a955e828f15f4966a96f217dbecdb2ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:59:51.229761 27856 tablet_replica.cc:439] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } } }
I20260808 22:59:51.229841 27856 tablet_replica.cc:442] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:59:51.229805 27833 tablet_replica.cc:439] T a347d157cd6944c88275313d8094bd5c P a955e828f15f4966a96f217dbecdb2ac: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } } }
I20260808 22:59:51.229877 27833 tablet_replica.cc:442] T a347d157cd6944c88275313d8094bd5c P a955e828f15f4966a96f217dbecdb2ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:59:51.230296 27836 tablet_replica.cc:439] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } } }
I20260808 22:59:51.230371 27836 tablet_replica.cc:442] T a347d157cd6944c88275313d8094bd5c P dafd6b7d77a444b6aaa6deb58300d9eb: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:59:51.230347 27865 mvcc.cc:205] Tried to move back new op lower bound from 7316398044063911936 to 7316398044014080000. Current Snapshot: MvccSnapshot[applied={T|T < 7316398044063911936}]
I20260808 22:59:51.230988 27835 tablet_replica.cc:439] T a347d157cd6944c88275313d8094bd5c P 08e21cbce6254c83bcddee69082c86fe: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } } }
I20260808 22:59:51.231071 27835 tablet_replica.cc:442] T a347d157cd6944c88275313d8094bd5c P 08e21cbce6254c83bcddee69082c86fe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:59:51.231525 27864 mvcc.cc:205] Tried to move back new op lower bound from 7316398044063911936 to 7316398044014080000. Current Snapshot: MvccSnapshot[applied={T|T < 7316398044063911936 or (T in {7316398044063911936})}]
W20260808 22:59:51.233990 27797 tablet.cc:2369] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:59:51.235287 27824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.232670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58386 (local address 127.23.194.2:40651)
0808 22:59:51.232707 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:51.232709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:51.232748 (+ 39us) server_negotiation.cc:407] Connection header received
0808 22:59:51.232823 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:51.232824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:51.232837 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:51.232890 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:51.233010 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.233543 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.234629 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.234808 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.234860 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.235009 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:51.235120 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:51.235173 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:51.235224 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 22:59:51.235240 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 22:59:51.239703 27822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.237717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:37945 (local address 127.23.194.3:40031)
0808 22:59:51.237759 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:51.237761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:51.237769 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:59:51.237809 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:51.237810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:51.237819 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:51.237862 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:51.237979 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.238415 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.239119 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.239294 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.239346 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.239503 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:51.239531 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:51.239570 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:51.239620 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 22:59:51.239638 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 22:59:51.239717 27584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.237653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:40031 (local address 127.23.194.2:37945)
0808 22:59:51.237713 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:51.237723 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:51.237761 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:51.237868 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:51.237869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:51.237870 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:59:51.237929 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.237930 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.238483 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.238484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.239043 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.239045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.239351 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.239353 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.239414 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.239427 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:59:51.239578 (+ 151us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:59:51.239580 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 22:59:51.239619 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 22:59:51.239658 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 22:59:51.242039 27281 raft_consensus.cc:1261] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe [term 1 FOLLOWER]: Refusing update from remote peer a955e828f15f4966a96f217dbecdb2ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:51.242128 27721 raft_consensus.cc:1261] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb [term 1 FOLLOWER]: Refusing update from remote peer a955e828f15f4966a96f217dbecdb2ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:51.242270 27840 consensus_queue.cc:1035] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:51.242370 27833 consensus_queue.cc:1035] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:51.247277 27863 mvcc.cc:205] Tried to move back new op lower bound from 7316398044125560832 to 7316398043346247680. Current Snapshot: MvccSnapshot[applied={T|T < 7316398044125560832}]
I20260808 22:59:51.250687 27824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.248153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:42859 (local address 127.23.194.2:40651)
0808 22:59:51.248190 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:51.248192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:51.248232 (+ 40us) server_negotiation.cc:407] Connection header received
0808 22:59:51.248284 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:51.248284 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:51.248291 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:51.248331 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:51.248437 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.248862 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.249977 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.250279 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.250339 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.250488 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:51.250519 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:51.250570 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:51.250605 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 22:59:51.250623 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 22:59:51.250694 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:51.248114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:40651 (local address 127.23.194.3:42859)
0808 22:59:51.248189 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:51.248198 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:51.248242 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:51.248335 (+ 93us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:51.248336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:51.248337 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:59:51.248388 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.248389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.248923 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.248924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.249885 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:51.249888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:51.250341 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:51.250341 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:51.250401 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:51.250414 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:59:51.250572 (+ 158us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:59:51.250573 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 22:59:51.250602 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 22:59:51.250633 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 22:59:51.259963 27033 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:59:51.260074 27033 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:59:51.260792 27033 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611: Sending DeleteTablet for 3 replicas of tablet c8b5f5c9a57742398bf9bbb1abdff58c
I20260808 22:59:51.261104 27244 tablet_service.cc:1514] Processing DeleteTablet for tablet c8b5f5c9a57742398bf9bbb1abdff58c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:51 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:38196
I20260808 22:59:51.261138 27466 tablet_service.cc:1514] Processing DeleteTablet for tablet c8b5f5c9a57742398bf9bbb1abdff58c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:51 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:58374
I20260808 22:59:51.261467 27892 tablet_replica.cc:330] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe: stopping tablet replica
I20260808 22:59:51.261468 27694 tablet_service.cc:1514] Processing DeleteTablet for tablet c8b5f5c9a57742398bf9bbb1abdff58c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:51 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:48214
I20260808 22:59:51.261577 27892 raft_consensus.cc:2229] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:59:51.261634 27892 raft_consensus.cc:2258] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:51.261865 27033 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:59:51.262009 27033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:59:51.262135 27893 tablet_replica.cc:330] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac: stopping tablet replica
I20260808 22:59:51.262291 27893 raft_consensus.cc:2229] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:59:51.262428 27893 raft_consensus.cc:2258] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:51.262887 27893 ts_tablet_manager.cc:1905] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:59:51.263298 27244 tablet_service.cc:1467] Processing CreateTablet for tablet 724bef734aed4251913b848ffc7e0a68 (DEFAULT_TABLE table=test-table [id=ca4fbdd89c3143359e318abff0ce014d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:51.263338 27466 tablet_service.cc:1467] Processing CreateTablet for tablet 724bef734aed4251913b848ffc7e0a68 (DEFAULT_TABLE table=test-table [id=ca4fbdd89c3143359e318abff0ce014d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:51.263408 27244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 724bef734aed4251913b848ffc7e0a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:51.263450 27466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 724bef734aed4251913b848ffc7e0a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:51.263707 27892 ts_tablet_manager.cc:1905] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:59:51.264775 27694 tablet_service.cc:1467] Processing CreateTablet for tablet 724bef734aed4251913b848ffc7e0a68 (DEFAULT_TABLE table=test-table [id=ca4fbdd89c3143359e318abff0ce014d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:51.264875 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 724bef734aed4251913b848ffc7e0a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:51.264864 27829 tablet_bootstrap.cc:492] T 724bef734aed4251913b848ffc7e0a68 P 08e21cbce6254c83bcddee69082c86fe: Bootstrap starting.
I20260808 22:59:51.265282 27829 tablet_bootstrap.cc:654] T 724bef734aed4251913b848ffc7e0a68 P 08e21cbce6254c83bcddee69082c86fe: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:51.265440 27827 tablet_bootstrap.cc:492] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac: Bootstrap starting.
I20260808 22:59:51.265552 27892 ts_tablet_manager.cc:1918] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:59:51.265628 27892 log.cc:1196] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-0/wal/wals/c8b5f5c9a57742398bf9bbb1abdff58c
I20260808 22:59:51.265865 27827 tablet_bootstrap.cc:654] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:51.265868 27892 ts_tablet_manager.cc:1939] T c8b5f5c9a57742398bf9bbb1abdff58c P 08e21cbce6254c83bcddee69082c86fe: Deleting consensus metadata
I20260808 22:59:51.266258 27011 catalog_manager.cc:4841] TS 08e21cbce6254c83bcddee69082c86fe (127.23.194.1:39133): tablet c8b5f5c9a57742398bf9bbb1abdff58c (table test-table [id=47251987dca543d3874ca9cab741d568]) successfully deleted
I20260808 22:59:51.266824 27893 ts_tablet_manager.cc:1918] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:59:51.266891 27893 log.cc:1196] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/c8b5f5c9a57742398bf9bbb1abdff58c
I20260808 22:59:51.267099 27828 tablet_bootstrap.cc:492] T 724bef734aed4251913b848ffc7e0a68 P dafd6b7d77a444b6aaa6deb58300d9eb: Bootstrap starting.
I20260808 22:59:51.267115 27893 ts_tablet_manager.cc:1939] T c8b5f5c9a57742398bf9bbb1abdff58c P a955e828f15f4966a96f217dbecdb2ac: Deleting consensus metadata
I20260808 22:59:51.267479 27828 tablet_bootstrap.cc:654] T 724bef734aed4251913b848ffc7e0a68 P dafd6b7d77a444b6aaa6deb58300d9eb: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:51.267462 27011 catalog_manager.cc:4841] TS a955e828f15f4966a96f217dbecdb2ac (127.23.194.2:40651): tablet c8b5f5c9a57742398bf9bbb1abdff58c (table test-table [id=47251987dca543d3874ca9cab741d568]) successfully deleted
I20260808 22:59:51.267728 27827 tablet_bootstrap.cc:492] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac: No bootstrap required, opened a new log
I20260808 22:59:51.267773 27827 ts_tablet_manager.cc:1397] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:59:51.267894 27827 raft_consensus.cc:348] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:51.267958 27827 raft_consensus.cc:374] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:51.267978 27827 raft_consensus.cc:729] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a955e828f15f4966a96f217dbecdb2ac, State: Initialized, Role: FOLLOWER
I20260808 22:59:51.268018 27827 consensus_queue.cc:260] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:51.268093 27827 ts_tablet_manager.cc:1428] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:51.268352 27828 tablet_bootstrap.cc:492] T 724bef734aed4251913b848ffc7e0a68 P dafd6b7d77a444b6aaa6deb58300d9eb: No bootstrap required, opened a new log
I20260808 22:59:51.268401 27828 ts_tablet_manager.cc:1397] T 724bef734aed4251913b848ffc7e0a68 P dafd6b7d77a444b6aaa6deb58300d9eb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:51.268544 27828 raft_consensus.cc:348] T 724bef734aed4251913b848ffc7e0a68 P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:51.268627 27828 raft_consensus.cc:374] T 724bef734aed4251913b848ffc7e0a68 P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:51.268654 27828 raft_consensus.cc:729] T 724bef734aed4251913b848ffc7e0a68 P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dafd6b7d77a444b6aaa6deb58300d9eb, State: Initialized, Role: FOLLOWER
I20260808 22:59:51.268692 27828 consensus_queue.cc:260] T 724bef734aed4251913b848ffc7e0a68 P dafd6b7d77a444b6aaa6deb58300d9eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:51.268778 27828 ts_tablet_manager.cc:1428] T 724bef734aed4251913b848ffc7e0a68 P dafd6b7d77a444b6aaa6deb58300d9eb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:51.273455 27895 tablet_replica.cc:330] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb: stopping tablet replica
I20260808 22:59:51.273571 27895 raft_consensus.cc:2229] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:59:51.273628 27895 raft_consensus.cc:2258] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:51.273756 27829 tablet_bootstrap.cc:492] T 724bef734aed4251913b848ffc7e0a68 P 08e21cbce6254c83bcddee69082c86fe: No bootstrap required, opened a new log
I20260808 22:59:51.273797 27829 ts_tablet_manager.cc:1397] T 724bef734aed4251913b848ffc7e0a68 P 08e21cbce6254c83bcddee69082c86fe: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20260808 22:59:51.273929 27829 raft_consensus.cc:348] T 724bef734aed4251913b848ffc7e0a68 P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:51.273986 27829 raft_consensus.cc:374] T 724bef734aed4251913b848ffc7e0a68 P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:51.274005 27829 raft_consensus.cc:729] T 724bef734aed4251913b848ffc7e0a68 P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e21cbce6254c83bcddee69082c86fe, State: Initialized, Role: FOLLOWER
I20260808 22:59:51.274062 27829 consensus_queue.cc:260] T 724bef734aed4251913b848ffc7e0a68 P 08e21cbce6254c83bcddee69082c86fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:51.274142 27829 ts_tablet_manager.cc:1428] T 724bef734aed4251913b848ffc7e0a68 P 08e21cbce6254c83bcddee69082c86fe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:51.274436 27895 ts_tablet_manager.cc:1905] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:59:51.275761 27895 ts_tablet_manager.cc:1918] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:59:51.275821 27895 log.cc:1196] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/c8b5f5c9a57742398bf9bbb1abdff58c
I20260808 22:59:51.276036 27895 ts_tablet_manager.cc:1939] T c8b5f5c9a57742398bf9bbb1abdff58c P dafd6b7d77a444b6aaa6deb58300d9eb: Deleting consensus metadata
I20260808 22:59:51.276378 27013 catalog_manager.cc:4841] TS dafd6b7d77a444b6aaa6deb58300d9eb (127.23.194.3:40031): tablet c8b5f5c9a57742398bf9bbb1abdff58c (table test-table [id=47251987dca543d3874ca9cab741d568]) successfully deleted
I20260808 22:59:51.399605 27888 raft_consensus.cc:482] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:59:51.399716 27888 raft_consensus.cc:504] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:51.399887 27888 leader_election.cc:288] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08e21cbce6254c83bcddee69082c86fe (127.23.194.1:39133), dafd6b7d77a444b6aaa6deb58300d9eb (127.23.194.3:40031)
I20260808 22:59:51.400131 27281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "724bef734aed4251913b848ffc7e0a68" candidate_uuid: "a955e828f15f4966a96f217dbecdb2ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08e21cbce6254c83bcddee69082c86fe" is_pre_election: true
I20260808 22:59:51.400130 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "724bef734aed4251913b848ffc7e0a68" candidate_uuid: "a955e828f15f4966a96f217dbecdb2ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" is_pre_election: true
I20260808 22:59:51.400223 27281 raft_consensus.cc:2454] T 724bef734aed4251913b848ffc7e0a68 P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a955e828f15f4966a96f217dbecdb2ac in term 0.
I20260808 22:59:51.400228 27721 raft_consensus.cc:2454] T 724bef734aed4251913b848ffc7e0a68 P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a955e828f15f4966a96f217dbecdb2ac in term 0.
I20260808 22:59:51.400394 27398 leader_election.cc:302] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a955e828f15f4966a96f217dbecdb2ac, dafd6b7d77a444b6aaa6deb58300d9eb; no voters:
I20260808 22:59:51.400507 27888 raft_consensus.cc:2781] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:59:51.400561 27888 raft_consensus.cc:482] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:59:51.400583 27888 raft_consensus.cc:3037] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:51.401227 27888 raft_consensus.cc:504] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:51.401341 27888 leader_election.cc:288] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [CANDIDATE]: Term 1 election: Requested vote from peers 08e21cbce6254c83bcddee69082c86fe (127.23.194.1:39133), dafd6b7d77a444b6aaa6deb58300d9eb (127.23.194.3:40031)
I20260808 22:59:51.401530 27281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "724bef734aed4251913b848ffc7e0a68" candidate_uuid: "a955e828f15f4966a96f217dbecdb2ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08e21cbce6254c83bcddee69082c86fe"
I20260808 22:59:51.401540 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "724bef734aed4251913b848ffc7e0a68" candidate_uuid: "a955e828f15f4966a96f217dbecdb2ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb"
I20260808 22:59:51.401599 27721 raft_consensus.cc:3037] T 724bef734aed4251913b848ffc7e0a68 P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:51.401595 27281 raft_consensus.cc:3037] T 724bef734aed4251913b848ffc7e0a68 P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:51.402158 27281 raft_consensus.cc:2454] T 724bef734aed4251913b848ffc7e0a68 P 08e21cbce6254c83bcddee69082c86fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a955e828f15f4966a96f217dbecdb2ac in term 1.
I20260808 22:59:51.402245 27721 raft_consensus.cc:2454] T 724bef734aed4251913b848ffc7e0a68 P dafd6b7d77a444b6aaa6deb58300d9eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a955e828f15f4966a96f217dbecdb2ac in term 1.
I20260808 22:59:51.402354 27396 leader_election.cc:302] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08e21cbce6254c83bcddee69082c86fe, a955e828f15f4966a96f217dbecdb2ac; no voters:
I20260808 22:59:51.402438 27888 raft_consensus.cc:2781] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:51.402495 27888 raft_consensus.cc:686] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [term 1 LEADER]: Becoming Leader. State: Replica: a955e828f15f4966a96f217dbecdb2ac, State: Running, Role: LEADER
I20260808 22:59:51.402549 27888 consensus_queue.cc:237] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:51.403139 27033 catalog_manager.cc:5495] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac reported cstate change: term changed from 0 to 1, leader changed from <none> to a955e828f15f4966a96f217dbecdb2ac (127.23.194.2). New cstate: current_term: 1 leader_uuid: "a955e828f15f4966a96f217dbecdb2ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260808 22:59:51.412170 27281 raft_consensus.cc:1261] T 724bef734aed4251913b848ffc7e0a68 P 08e21cbce6254c83bcddee69082c86fe [term 1 FOLLOWER]: Refusing update from remote peer a955e828f15f4966a96f217dbecdb2ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:51.412200 27721 raft_consensus.cc:1261] T 724bef734aed4251913b848ffc7e0a68 P dafd6b7d77a444b6aaa6deb58300d9eb [term 1 FOLLOWER]: Refusing update from remote peer a955e828f15f4966a96f217dbecdb2ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:51.412370 27840 consensus_queue.cc:1035] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:51.412528 27840 consensus_queue.cc:1035] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 22:59:51.416646 27865 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 22:59:52.933905 27033 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:59:52.934049 27033 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:59:52.934952 27033 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611: Sending DeleteTablet for 3 replicas of tablet 724bef734aed4251913b848ffc7e0a68
I20260808 22:59:52.935237 27694 tablet_service.cc:1514] Processing DeleteTablet for tablet 724bef734aed4251913b848ffc7e0a68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:52 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:48214
I20260808 22:59:52.935249 27244 tablet_service.cc:1514] Processing DeleteTablet for tablet 724bef734aed4251913b848ffc7e0a68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:52 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:38196
I20260808 22:59:52.935567 27941 tablet_replica.cc:330] T 724bef734aed4251913b848ffc7e0a68 P 08e21cbce6254c83bcddee69082c86fe: stopping tablet replica
I20260808 22:59:52.935633 27941 raft_consensus.cc:2229] T 724bef734aed4251913b848ffc7e0a68 P 08e21cbce6254c83bcddee69082c86fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:59:52.935696 27941 raft_consensus.cc:2258] T 724bef734aed4251913b848ffc7e0a68 P 08e21cbce6254c83bcddee69082c86fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:52.935791 27940 tablet_replica.cc:330] T 724bef734aed4251913b848ffc7e0a68 P dafd6b7d77a444b6aaa6deb58300d9eb: stopping tablet replica
I20260808 22:59:52.935845 27940 raft_consensus.cc:2229] T 724bef734aed4251913b848ffc7e0a68 P dafd6b7d77a444b6aaa6deb58300d9eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:59:52.935907 27940 raft_consensus.cc:2258] T 724bef734aed4251913b848ffc7e0a68 P dafd6b7d77a444b6aaa6deb58300d9eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:52.935945 27941 ts_tablet_manager.cc:1905] T 724bef734aed4251913b848ffc7e0a68 P 08e21cbce6254c83bcddee69082c86fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:59:52.936131 27940 ts_tablet_manager.cc:1905] T 724bef734aed4251913b848ffc7e0a68 P dafd6b7d77a444b6aaa6deb58300d9eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:59:52.937327 27466 tablet_service.cc:1514] Processing DeleteTablet for tablet 724bef734aed4251913b848ffc7e0a68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:52 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:58374
I20260808 22:59:52.937552 27940 ts_tablet_manager.cc:1918] T 724bef734aed4251913b848ffc7e0a68 P dafd6b7d77a444b6aaa6deb58300d9eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 22:59:52.937660 27940 log.cc:1196] T 724bef734aed4251913b848ffc7e0a68 P dafd6b7d77a444b6aaa6deb58300d9eb: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/724bef734aed4251913b848ffc7e0a68
I20260808 22:59:52.937729 27941 ts_tablet_manager.cc:1918] T 724bef734aed4251913b848ffc7e0a68 P 08e21cbce6254c83bcddee69082c86fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 22:59:52.937860 27941 log.cc:1196] T 724bef734aed4251913b848ffc7e0a68 P 08e21cbce6254c83bcddee69082c86fe: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-0/wal/wals/724bef734aed4251913b848ffc7e0a68
I20260808 22:59:52.937923 27940 ts_tablet_manager.cc:1939] T 724bef734aed4251913b848ffc7e0a68 P dafd6b7d77a444b6aaa6deb58300d9eb: Deleting consensus metadata
I20260808 22:59:52.938208 27013 catalog_manager.cc:4841] TS dafd6b7d77a444b6aaa6deb58300d9eb (127.23.194.3:40031): tablet 724bef734aed4251913b848ffc7e0a68 (table test-table [id=ca4fbdd89c3143359e318abff0ce014d]) successfully deleted
I20260808 22:59:52.938227 27941 ts_tablet_manager.cc:1939] T 724bef734aed4251913b848ffc7e0a68 P 08e21cbce6254c83bcddee69082c86fe: Deleting consensus metadata
I20260808 22:59:52.938402 27943 tablet_replica.cc:330] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac: stopping tablet replica
I20260808 22:59:52.938503 27943 raft_consensus.cc:2229] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:59:52.938577 27011 catalog_manager.cc:4841] TS 08e21cbce6254c83bcddee69082c86fe (127.23.194.1:39133): tablet 724bef734aed4251913b848ffc7e0a68 (table test-table [id=ca4fbdd89c3143359e318abff0ce014d]) successfully deleted
I20260808 22:59:52.938603 27943 raft_consensus.cc:2258] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:52.938995 27943 ts_tablet_manager.cc:1905] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:59:52.940526 27943 ts_tablet_manager.cc:1918] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 22:59:52.940588 27943 log.cc:1196] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/724bef734aed4251913b848ffc7e0a68
I20260808 22:59:52.940837 27943 ts_tablet_manager.cc:1939] T 724bef734aed4251913b848ffc7e0a68 P a955e828f15f4966a96f217dbecdb2ac: Deleting consensus metadata
I20260808 22:59:52.941120 27011 catalog_manager.cc:4841] TS a955e828f15f4966a96f217dbecdb2ac (127.23.194.2:40651): tablet 724bef734aed4251913b848ffc7e0a68 (table test-table [id=ca4fbdd89c3143359e318abff0ce014d]) successfully deleted
Aug 08 22:59:52 dist-test-slave-f0tn krb5kdc[26961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229992, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 22:59:52 dist-test-slave-f0tn krb5kdc[26961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229992, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:52.963112 27956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:52.952121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32838 (local address 127.23.194.62:45317)
0808 22:59:52.952273 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:52.952274 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:52.952287 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:59:52.952346 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:52.952347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:52.952360 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:52.952418 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:52.952937 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:52.953515 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:52.954220 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:52.954417 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:52.954467 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:52.960921 (+ 6454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:52.960922 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:52.960923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:52.960930 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:52.962312 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:52.962572 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:52.962572 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:52.962572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:52.962620 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:52.962742 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:52.962743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:52.962743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:52.962833 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:52.962900 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:52.962969 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 22:59:52.963026 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":103,"threads_started":1}
I20260808 22:59:52.964409 27033 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32838:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:59:52.964553 27033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:59:52.965806 27694 tablet_service.cc:1467] Processing CreateTablet for tablet a408628ea5d04d708cf17b4ed9683bde (DEFAULT_TABLE table=test-table [id=55dd25d89a99481dae71293942024153]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:52.965811 27244 tablet_service.cc:1467] Processing CreateTablet for tablet a408628ea5d04d708cf17b4ed9683bde (DEFAULT_TABLE table=test-table [id=55dd25d89a99481dae71293942024153]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:52.965829 27466 tablet_service.cc:1467] Processing CreateTablet for tablet a408628ea5d04d708cf17b4ed9683bde (DEFAULT_TABLE table=test-table [id=55dd25d89a99481dae71293942024153]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:52.965945 27466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a408628ea5d04d708cf17b4ed9683bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:52.965945 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a408628ea5d04d708cf17b4ed9683bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:52.965951 27244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a408628ea5d04d708cf17b4ed9683bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:52.967199 27957 tablet_bootstrap.cc:492] T a408628ea5d04d708cf17b4ed9683bde P 08e21cbce6254c83bcddee69082c86fe: Bootstrap starting.
I20260808 22:59:52.967599 27957 tablet_bootstrap.cc:654] T a408628ea5d04d708cf17b4ed9683bde P 08e21cbce6254c83bcddee69082c86fe: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:52.967741 27958 tablet_bootstrap.cc:492] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac: Bootstrap starting.
I20260808 22:59:52.968183 27958 tablet_bootstrap.cc:654] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:52.968623 27957 tablet_bootstrap.cc:492] T a408628ea5d04d708cf17b4ed9683bde P 08e21cbce6254c83bcddee69082c86fe: No bootstrap required, opened a new log
I20260808 22:59:52.968726 27957 ts_tablet_manager.cc:1397] T a408628ea5d04d708cf17b4ed9683bde P 08e21cbce6254c83bcddee69082c86fe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:59:52.968868 27957 raft_consensus.cc:348] T a408628ea5d04d708cf17b4ed9683bde P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:52.968967 27957 raft_consensus.cc:374] T a408628ea5d04d708cf17b4ed9683bde P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:52.968997 27957 raft_consensus.cc:729] T a408628ea5d04d708cf17b4ed9683bde P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e21cbce6254c83bcddee69082c86fe, State: Initialized, Role: FOLLOWER
I20260808 22:59:52.969051 27964 tablet_bootstrap.cc:492] T a408628ea5d04d708cf17b4ed9683bde P dafd6b7d77a444b6aaa6deb58300d9eb: Bootstrap starting.
I20260808 22:59:52.969059 27957 consensus_queue.cc:260] T a408628ea5d04d708cf17b4ed9683bde P 08e21cbce6254c83bcddee69082c86fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:52.969218 27957 ts_tablet_manager.cc:1428] T a408628ea5d04d708cf17b4ed9683bde P 08e21cbce6254c83bcddee69082c86fe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:52.969465 27964 tablet_bootstrap.cc:654] T a408628ea5d04d708cf17b4ed9683bde P dafd6b7d77a444b6aaa6deb58300d9eb: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:52.969771 27958 tablet_bootstrap.cc:492] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac: No bootstrap required, opened a new log
I20260808 22:59:52.969856 27958 ts_tablet_manager.cc:1397] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260808 22:59:52.969985 27958 raft_consensus.cc:348] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:52.970057 27958 raft_consensus.cc:374] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:52.970074 27958 raft_consensus.cc:729] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a955e828f15f4966a96f217dbecdb2ac, State: Initialized, Role: FOLLOWER
I20260808 22:59:52.970121 27958 consensus_queue.cc:260] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:52.970258 27958 ts_tablet_manager.cc:1428] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:52.970356 27964 tablet_bootstrap.cc:492] T a408628ea5d04d708cf17b4ed9683bde P dafd6b7d77a444b6aaa6deb58300d9eb: No bootstrap required, opened a new log
I20260808 22:59:52.970429 27964 ts_tablet_manager.cc:1397] T a408628ea5d04d708cf17b4ed9683bde P dafd6b7d77a444b6aaa6deb58300d9eb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:59:52.970557 27964 raft_consensus.cc:348] T a408628ea5d04d708cf17b4ed9683bde P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:52.970628 27964 raft_consensus.cc:374] T a408628ea5d04d708cf17b4ed9683bde P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:52.970647 27964 raft_consensus.cc:729] T a408628ea5d04d708cf17b4ed9683bde P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dafd6b7d77a444b6aaa6deb58300d9eb, State: Initialized, Role: FOLLOWER
I20260808 22:59:52.970708 27964 consensus_queue.cc:260] T a408628ea5d04d708cf17b4ed9683bde P dafd6b7d77a444b6aaa6deb58300d9eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:52.970824 27964 ts_tablet_manager.cc:1428] T a408628ea5d04d708cf17b4ed9683bde P dafd6b7d77a444b6aaa6deb58300d9eb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:53.198681 27929 raft_consensus.cc:482] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:59:53.198777 27929 raft_consensus.cc:504] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:53.198936 27929 leader_election.cc:288] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08e21cbce6254c83bcddee69082c86fe (127.23.194.1:39133), dafd6b7d77a444b6aaa6deb58300d9eb (127.23.194.3:40031)
I20260808 22:59:53.199172 27281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a408628ea5d04d708cf17b4ed9683bde" candidate_uuid: "a955e828f15f4966a96f217dbecdb2ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08e21cbce6254c83bcddee69082c86fe" is_pre_election: true
I20260808 22:59:53.199172 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a408628ea5d04d708cf17b4ed9683bde" candidate_uuid: "a955e828f15f4966a96f217dbecdb2ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" is_pre_election: true
I20260808 22:59:53.199275 27721 raft_consensus.cc:2454] T a408628ea5d04d708cf17b4ed9683bde P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a955e828f15f4966a96f217dbecdb2ac in term 0.
I20260808 22:59:53.199275 27281 raft_consensus.cc:2454] T a408628ea5d04d708cf17b4ed9683bde P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a955e828f15f4966a96f217dbecdb2ac in term 0.
I20260808 22:59:53.199424 27398 leader_election.cc:302] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a955e828f15f4966a96f217dbecdb2ac, dafd6b7d77a444b6aaa6deb58300d9eb; no voters:
I20260808 22:59:53.199563 27929 raft_consensus.cc:2781] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:59:53.199610 27929 raft_consensus.cc:482] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:59:53.199625 27929 raft_consensus.cc:3037] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:53.200392 27929 raft_consensus.cc:504] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:53.200520 27929 leader_election.cc:288] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [CANDIDATE]: Term 1 election: Requested vote from peers 08e21cbce6254c83bcddee69082c86fe (127.23.194.1:39133), dafd6b7d77a444b6aaa6deb58300d9eb (127.23.194.3:40031)
I20260808 22:59:53.200659 27281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a408628ea5d04d708cf17b4ed9683bde" candidate_uuid: "a955e828f15f4966a96f217dbecdb2ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08e21cbce6254c83bcddee69082c86fe"
I20260808 22:59:53.200659 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a408628ea5d04d708cf17b4ed9683bde" candidate_uuid: "a955e828f15f4966a96f217dbecdb2ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb"
I20260808 22:59:53.200745 27281 raft_consensus.cc:3037] T a408628ea5d04d708cf17b4ed9683bde P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:53.200745 27721 raft_consensus.cc:3037] T a408628ea5d04d708cf17b4ed9683bde P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:53.201507 27281 raft_consensus.cc:2454] T a408628ea5d04d708cf17b4ed9683bde P 08e21cbce6254c83bcddee69082c86fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a955e828f15f4966a96f217dbecdb2ac in term 1.
I20260808 22:59:53.201516 27721 raft_consensus.cc:2454] T a408628ea5d04d708cf17b4ed9683bde P dafd6b7d77a444b6aaa6deb58300d9eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a955e828f15f4966a96f217dbecdb2ac in term 1.
I20260808 22:59:53.201670 27396 leader_election.cc:302] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08e21cbce6254c83bcddee69082c86fe, a955e828f15f4966a96f217dbecdb2ac; no voters:
I20260808 22:59:53.201781 27929 raft_consensus.cc:2781] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:53.201833 27929 raft_consensus.cc:686] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [term 1 LEADER]: Becoming Leader. State: Replica: a955e828f15f4966a96f217dbecdb2ac, State: Running, Role: LEADER
I20260808 22:59:53.201875 27929 consensus_queue.cc:237] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:53.202517 27031 catalog_manager.cc:5495] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac reported cstate change: term changed from 0 to 1, leader changed from <none> to a955e828f15f4966a96f217dbecdb2ac (127.23.194.2). New cstate: current_term: 1 leader_uuid: "a955e828f15f4966a96f217dbecdb2ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260808 22:59:53.252835 27971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:53.250630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58392 (local address 127.23.194.2:40651)
0808 22:59:53.250799 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:53.250801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:53.250807 (+ 6us) server_negotiation.cc:407] Connection header received
0808 22:59:53.250826 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:53.250826 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:53.250836 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:53.250886 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:53.251004 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:53.251519 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:53.252260 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:53.252450 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:53.252501 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:53.252625 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:53.252669 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:53.252700 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:53.252734 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 22:59:53.252751 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":112,"threads_started":1}
I20260808 22:59:53.254855 27721 raft_consensus.cc:1261] T a408628ea5d04d708cf17b4ed9683bde P dafd6b7d77a444b6aaa6deb58300d9eb [term 1 FOLLOWER]: Refusing update from remote peer a955e828f15f4966a96f217dbecdb2ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:53.254984 27281 raft_consensus.cc:1261] T a408628ea5d04d708cf17b4ed9683bde P 08e21cbce6254c83bcddee69082c86fe [term 1 FOLLOWER]: Refusing update from remote peer a955e828f15f4966a96f217dbecdb2ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:53.255067 27929 consensus_queue.cc:1035] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:53.255364 27929 consensus_queue.cc:1035] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:53.264604 27031 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:59:53.264693 27031 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:59:53.265237 27031 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611: Sending DeleteTablet for 3 replicas of tablet a408628ea5d04d708cf17b4ed9683bde
I20260808 22:59:53.265489 27244 tablet_service.cc:1514] Processing DeleteTablet for tablet a408628ea5d04d708cf17b4ed9683bde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:53 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:38196
I20260808 22:59:53.265522 27466 tablet_service.cc:1514] Processing DeleteTablet for tablet a408628ea5d04d708cf17b4ed9683bde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:53 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:58374
I20260808 22:59:53.265595 27941 tablet_replica.cc:330] T a408628ea5d04d708cf17b4ed9683bde P 08e21cbce6254c83bcddee69082c86fe: stopping tablet replica
I20260808 22:59:53.265590 27694 tablet_service.cc:1514] Processing DeleteTablet for tablet a408628ea5d04d708cf17b4ed9683bde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:53 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:48214
I20260808 22:59:53.265655 27941 raft_consensus.cc:2229] T a408628ea5d04d708cf17b4ed9683bde P 08e21cbce6254c83bcddee69082c86fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:59:53.265667 27940 tablet_replica.cc:330] T a408628ea5d04d708cf17b4ed9683bde P dafd6b7d77a444b6aaa6deb58300d9eb: stopping tablet replica
I20260808 22:59:53.265676 27943 tablet_replica.cc:330] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac: stopping tablet replica
I20260808 22:59:53.265692 27941 raft_consensus.cc:2258] T a408628ea5d04d708cf17b4ed9683bde P 08e21cbce6254c83bcddee69082c86fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:53.265714 27940 raft_consensus.cc:2229] T a408628ea5d04d708cf17b4ed9683bde P dafd6b7d77a444b6aaa6deb58300d9eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:59:53.265722 27943 raft_consensus.cc:2229] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:59:53.265751 27940 raft_consensus.cc:2258] T a408628ea5d04d708cf17b4ed9683bde P dafd6b7d77a444b6aaa6deb58300d9eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:53.265769 27943 raft_consensus.cc:2258] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:53.265908 27031 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32838:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:59:53.266048 27031 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:59:53.266057 27940 ts_tablet_manager.cc:1905] T a408628ea5d04d708cf17b4ed9683bde P dafd6b7d77a444b6aaa6deb58300d9eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:59:53.266363 27941 ts_tablet_manager.cc:1905] T a408628ea5d04d708cf17b4ed9683bde P 08e21cbce6254c83bcddee69082c86fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:59:53.267462 27940 ts_tablet_manager.cc:1918] T a408628ea5d04d708cf17b4ed9683bde P dafd6b7d77a444b6aaa6deb58300d9eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:59:53.267462 27941 ts_tablet_manager.cc:1918] T a408628ea5d04d708cf17b4ed9683bde P 08e21cbce6254c83bcddee69082c86fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:59:53.267477 27943 ts_tablet_manager.cc:1905] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:59:53.267518 27941 log.cc:1196] T a408628ea5d04d708cf17b4ed9683bde P 08e21cbce6254c83bcddee69082c86fe: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-0/wal/wals/a408628ea5d04d708cf17b4ed9683bde
I20260808 22:59:53.267518 27940 log.cc:1196] T a408628ea5d04d708cf17b4ed9683bde P dafd6b7d77a444b6aaa6deb58300d9eb: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/a408628ea5d04d708cf17b4ed9683bde
I20260808 22:59:53.267738 27940 ts_tablet_manager.cc:1939] T a408628ea5d04d708cf17b4ed9683bde P dafd6b7d77a444b6aaa6deb58300d9eb: Deleting consensus metadata
I20260808 22:59:53.267738 27941 ts_tablet_manager.cc:1939] T a408628ea5d04d708cf17b4ed9683bde P 08e21cbce6254c83bcddee69082c86fe: Deleting consensus metadata
I20260808 22:59:53.267992 27011 catalog_manager.cc:4841] TS 08e21cbce6254c83bcddee69082c86fe (127.23.194.1:39133): tablet a408628ea5d04d708cf17b4ed9683bde (table test-table [id=55dd25d89a99481dae71293942024153]) successfully deleted
I20260808 22:59:53.267998 27013 catalog_manager.cc:4841] TS dafd6b7d77a444b6aaa6deb58300d9eb (127.23.194.3:40031): tablet a408628ea5d04d708cf17b4ed9683bde (table test-table [id=55dd25d89a99481dae71293942024153]) successfully deleted
I20260808 22:59:53.268694 27943 ts_tablet_manager.cc:1918] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:59:53.268743 27943 log.cc:1196] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/a408628ea5d04d708cf17b4ed9683bde
I20260808 22:59:53.268910 27943 ts_tablet_manager.cc:1939] T a408628ea5d04d708cf17b4ed9683bde P a955e828f15f4966a96f217dbecdb2ac: Deleting consensus metadata
I20260808 22:59:53.269191 27011 catalog_manager.cc:4841] TS a955e828f15f4966a96f217dbecdb2ac (127.23.194.2:40651): tablet a408628ea5d04d708cf17b4ed9683bde (table test-table [id=55dd25d89a99481dae71293942024153]) successfully deleted
I20260808 22:59:53.269707 27244 tablet_service.cc:1467] Processing CreateTablet for tablet 660255707c10469a8248ec0520cf875b (DEFAULT_TABLE table=test-table [id=17f3d0df58e744a1b0e19e68e3e4976d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:53.269727 27694 tablet_service.cc:1467] Processing CreateTablet for tablet 660255707c10469a8248ec0520cf875b (DEFAULT_TABLE table=test-table [id=17f3d0df58e744a1b0e19e68e3e4976d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:53.269727 27466 tablet_service.cc:1467] Processing CreateTablet for tablet 660255707c10469a8248ec0520cf875b (DEFAULT_TABLE table=test-table [id=17f3d0df58e744a1b0e19e68e3e4976d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:59:53.269817 27244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 660255707c10469a8248ec0520cf875b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:53.269829 27466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 660255707c10469a8248ec0520cf875b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:53.269827 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 660255707c10469a8248ec0520cf875b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:53.270591 27957 tablet_bootstrap.cc:492] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe: Bootstrap starting.
I20260808 22:59:53.270936 27964 tablet_bootstrap.cc:492] T 660255707c10469a8248ec0520cf875b P dafd6b7d77a444b6aaa6deb58300d9eb: Bootstrap starting.
I20260808 22:59:53.270969 27957 tablet_bootstrap.cc:654] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:53.271065 27958 tablet_bootstrap.cc:492] T 660255707c10469a8248ec0520cf875b P a955e828f15f4966a96f217dbecdb2ac: Bootstrap starting.
I20260808 22:59:53.271291 27964 tablet_bootstrap.cc:654] T 660255707c10469a8248ec0520cf875b P dafd6b7d77a444b6aaa6deb58300d9eb: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:53.271365 27958 tablet_bootstrap.cc:654] T 660255707c10469a8248ec0520cf875b P a955e828f15f4966a96f217dbecdb2ac: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:53.271865 27958 tablet_bootstrap.cc:492] T 660255707c10469a8248ec0520cf875b P a955e828f15f4966a96f217dbecdb2ac: No bootstrap required, opened a new log
I20260808 22:59:53.271917 27958 ts_tablet_manager.cc:1397] T 660255707c10469a8248ec0520cf875b P a955e828f15f4966a96f217dbecdb2ac: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:59:53.271977 27964 tablet_bootstrap.cc:492] T 660255707c10469a8248ec0520cf875b P dafd6b7d77a444b6aaa6deb58300d9eb: No bootstrap required, opened a new log
I20260808 22:59:53.272020 27964 ts_tablet_manager.cc:1397] T 660255707c10469a8248ec0520cf875b P dafd6b7d77a444b6aaa6deb58300d9eb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:59:53.272028 27958 raft_consensus.cc:348] T 660255707c10469a8248ec0520cf875b P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:53.272099 27958 raft_consensus.cc:374] T 660255707c10469a8248ec0520cf875b P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:53.272125 27958 raft_consensus.cc:729] T 660255707c10469a8248ec0520cf875b P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a955e828f15f4966a96f217dbecdb2ac, State: Initialized, Role: FOLLOWER
I20260808 22:59:53.272169 27958 consensus_queue.cc:260] T 660255707c10469a8248ec0520cf875b P a955e828f15f4966a96f217dbecdb2ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:53.272148 27964 raft_consensus.cc:348] T 660255707c10469a8248ec0520cf875b P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:53.272217 27964 raft_consensus.cc:374] T 660255707c10469a8248ec0520cf875b P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:53.272244 27964 raft_consensus.cc:729] T 660255707c10469a8248ec0520cf875b P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dafd6b7d77a444b6aaa6deb58300d9eb, State: Initialized, Role: FOLLOWER
I20260808 22:59:53.272267 27958 ts_tablet_manager.cc:1428] T 660255707c10469a8248ec0520cf875b P a955e828f15f4966a96f217dbecdb2ac: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:53.272289 27964 consensus_queue.cc:260] T 660255707c10469a8248ec0520cf875b P dafd6b7d77a444b6aaa6deb58300d9eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:53.272369 27964 ts_tablet_manager.cc:1428] T 660255707c10469a8248ec0520cf875b P dafd6b7d77a444b6aaa6deb58300d9eb: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 22:59:53.310376 27957 tablet_bootstrap.cc:492] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe: No bootstrap required, opened a new log
I20260808 22:59:53.310444 27957 ts_tablet_manager.cc:1397] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe: Time spent bootstrapping tablet: real 0.040s user 0.001s sys 0.000s
I20260808 22:59:53.310801 27957 raft_consensus.cc:348] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:53.310884 27957 raft_consensus.cc:374] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:53.310897 27957 raft_consensus.cc:729] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e21cbce6254c83bcddee69082c86fe, State: Initialized, Role: FOLLOWER
I20260808 22:59:53.310951 27957 consensus_queue.cc:260] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:53.311058 27957 ts_tablet_manager.cc:1428] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 22:59:53.616811 27966 raft_consensus.cc:482] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:59:53.616918 27966 raft_consensus.cc:504] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:53.617244 27966 leader_election.cc:288] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a955e828f15f4966a96f217dbecdb2ac (127.23.194.2:40651), dafd6b7d77a444b6aaa6deb58300d9eb (127.23.194.3:40031)
I20260808 22:59:53.621236 27989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:53.617502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:50671 (local address 127.23.194.3:40031)
0808 22:59:53.617612 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:53.617614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:53.617749 (+ 135us) server_negotiation.cc:407] Connection header received
0808 22:59:53.617784 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:53.617785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:53.617794 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:53.617866 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:53.618134 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:53.618853 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:53.620315 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:53.620896 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:53.620957 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:53.620965 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:53.621116 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 22:59:53.621173 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":67,"threads_started":1}
I20260808 22:59:53.621228 27971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:53.617498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:41161 (local address 127.23.194.2:40651)
0808 22:59:53.617553 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:53.617555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:53.617687 (+ 132us) server_negotiation.cc:407] Connection header received
0808 22:59:53.617771 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:53.617772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:53.617782 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:53.617839 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:59:53.618017 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:53.618780 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:53.620318 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:53.620878 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:53.620951 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:53.620961 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:53.621116 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 22:59:53.621156 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 22:59:53.621268 27987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:53.617450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:40651 (local address 127.23.194.1:41161)
0808 22:59:53.617595 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:53.617606 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:53.617680 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:53.617850 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:53.617851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:53.617853 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:53.617944 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:53.617946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:53.618861 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:53.618862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:53.620243 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:53.620245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:53.620964 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:53.620965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:53.621049 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:53.621055 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:59:53.621104 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 22:59:53.621153 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":60,"thread_start_us":86,"threads_started":1}
I20260808 22:59:53.621268 27988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:53.617450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:40031 (local address 127.23.194.1:50671)
0808 22:59:53.617683 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:53.617692 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:53.617744 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:53.617871 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:53.617872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:53.617873 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:59:53.618069 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:53.618072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:53.619091 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:53.619092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:53.620234 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:53.620239 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:53.620965 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:53.620966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:53.621049 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:53.621054 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:59:53.621104 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 22:59:53.621151 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":6,"thread_start_us":79,"threads_started":1}
I20260808 22:59:53.621547 27501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "660255707c10469a8248ec0520cf875b" candidate_uuid: "08e21cbce6254c83bcddee69082c86fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a955e828f15f4966a96f217dbecdb2ac" is_pre_election: true
I20260808 22:59:53.621634 27501 raft_consensus.cc:2454] T 660255707c10469a8248ec0520cf875b P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08e21cbce6254c83bcddee69082c86fe in term 0.
I20260808 22:59:53.621750 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "660255707c10469a8248ec0520cf875b" candidate_uuid: "08e21cbce6254c83bcddee69082c86fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" is_pre_election: true
I20260808 22:59:53.621842 27721 raft_consensus.cc:2454] T 660255707c10469a8248ec0520cf875b P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08e21cbce6254c83bcddee69082c86fe in term 0.
I20260808 22:59:53.621852 27166 leader_election.cc:302] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08e21cbce6254c83bcddee69082c86fe, a955e828f15f4966a96f217dbecdb2ac; no voters:
I20260808 22:59:53.621987 27966 raft_consensus.cc:2781] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:59:53.622025 27966 raft_consensus.cc:482] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:59:53.622040 27966 raft_consensus.cc:3037] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:53.623138 27966 raft_consensus.cc:504] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:53.623262 27966 leader_election.cc:288] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe [CANDIDATE]: Term 1 election: Requested vote from peers a955e828f15f4966a96f217dbecdb2ac (127.23.194.2:40651), dafd6b7d77a444b6aaa6deb58300d9eb (127.23.194.3:40031)
I20260808 22:59:53.623384 27501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "660255707c10469a8248ec0520cf875b" candidate_uuid: "08e21cbce6254c83bcddee69082c86fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a955e828f15f4966a96f217dbecdb2ac"
I20260808 22:59:53.623397 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "660255707c10469a8248ec0520cf875b" candidate_uuid: "08e21cbce6254c83bcddee69082c86fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb"
I20260808 22:59:53.623464 27721 raft_consensus.cc:3037] T 660255707c10469a8248ec0520cf875b P dafd6b7d77a444b6aaa6deb58300d9eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:53.623464 27501 raft_consensus.cc:3037] T 660255707c10469a8248ec0520cf875b P a955e828f15f4966a96f217dbecdb2ac [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:53.624207 27721 raft_consensus.cc:2454] T 660255707c10469a8248ec0520cf875b P dafd6b7d77a444b6aaa6deb58300d9eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08e21cbce6254c83bcddee69082c86fe in term 1.
I20260808 22:59:53.624207 27501 raft_consensus.cc:2454] T 660255707c10469a8248ec0520cf875b P a955e828f15f4966a96f217dbecdb2ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08e21cbce6254c83bcddee69082c86fe in term 1.
I20260808 22:59:53.624367 27166 leader_election.cc:302] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08e21cbce6254c83bcddee69082c86fe, a955e828f15f4966a96f217dbecdb2ac; no voters:
I20260808 22:59:53.624449 27966 raft_consensus.cc:2781] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:53.624607 27966 raft_consensus.cc:686] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe [term 1 LEADER]: Becoming Leader. State: Replica: 08e21cbce6254c83bcddee69082c86fe, State: Running, Role: LEADER
I20260808 22:59:53.624683 27966 consensus_queue.cc:237] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } }
I20260808 22:59:53.625335 27033 catalog_manager.cc:5495] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 08e21cbce6254c83bcddee69082c86fe (127.23.194.1). New cstate: current_term: 1 leader_uuid: "08e21cbce6254c83bcddee69082c86fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08e21cbce6254c83bcddee69082c86fe" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 39133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260808 22:59:53.711705 27995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:53.709150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38206 (local address 127.23.194.1:39133)
0808 22:59:53.709426 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:53.709428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:53.709436 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:59:53.709478 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:53.709479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:53.709499 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:53.709573 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:53.709715 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:53.710209 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:53.710914 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:53.711151 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:53.711207 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:53.711367 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:53.711482 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:53.711546 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:53.711594 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 22:59:53.711614 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":181,"threads_started":1}
I20260808 22:59:53.716707 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:53.714363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:40031 (local address 127.23.194.1:34829)
0808 22:59:53.714551 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:53.714562 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:53.714616 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:53.714765 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:53.714766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:53.714767 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:59:53.714822 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:53.714824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:53.715391 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:53.715392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:53.715946 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:53.715948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:53.716316 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:53.716318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:53.716372 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:53.716385 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:59:53.716541 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:59:53.716543 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 22:59:53.716583 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 22:59:53.716625 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
I20260808 22:59:53.716692 27989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:53.714458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:34829 (local address 127.23.194.3:40031)
0808 22:59:53.714520 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:53.714522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:53.714612 (+ 90us) server_negotiation.cc:407] Connection header received
0808 22:59:53.714698 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:53.714699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:53.714710 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:53.714757 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:53.714879 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:53.715322 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:53.716023 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:53.716232 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:53.716285 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:53.716453 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:53.716489 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:53.716530 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:53.716584 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 22:59:53.716607 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:59:53.718829 27501 raft_consensus.cc:1261] T 660255707c10469a8248ec0520cf875b P a955e828f15f4966a96f217dbecdb2ac [term 1 FOLLOWER]: Refusing update from remote peer 08e21cbce6254c83bcddee69082c86fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:53.718851 27721 raft_consensus.cc:1261] T 660255707c10469a8248ec0520cf875b P dafd6b7d77a444b6aaa6deb58300d9eb [term 1 FOLLOWER]: Refusing update from remote peer 08e21cbce6254c83bcddee69082c86fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:59:53.719225 27990 consensus_queue.cc:1035] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "a955e828f15f4966a96f217dbecdb2ac" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:53.719358 27966 consensus_queue.cc:1035] T 660255707c10469a8248ec0520cf875b P 08e21cbce6254c83bcddee69082c86fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "dafd6b7d77a444b6aaa6deb58300d9eb" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:59:53.726703 27995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:53.724403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:37955 (local address 127.23.194.1:39133)
0808 22:59:53.724456 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:53.724458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:53.724556 (+ 98us) server_negotiation.cc:407] Connection header received
0808 22:59:53.724607 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:53.724608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:53.724619 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:53.724680 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:59:53.724838 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:53.725272 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:53.726109 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:53.726371 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:53.726428 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:53.726542 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:59:53.726567 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:59:53.726609 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:53.726640 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 22:59:53.726656 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 22:59:53.726768 28006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:53.724333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:39133 (local address 127.23.194.3:37955)
0808 22:59:53.724486 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:53.724497 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:53.724551 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:53.724684 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:53.724685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:53.724686 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:59:53.724793 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:53.724794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:53.725330 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:53.725331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:53.726034 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:53.726036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:53.726439 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:53.726440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:53.726493 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:53.726496 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:59:53.726610 (+ 114us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:59:53.726611 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 22:59:53.726642 (+ 31us) client_negotiation.cc:240] Negotiation successful
0808 22:59:53.726676 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":102,"threads_started":1}
W20260808 22:59:53.727236 27911 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 22:59:55.250731 27032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:59:55.250825 27032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:59:55.251657 27032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 865550ce39cd438dba96f8cbe1863611: Sending DeleteTablet for 3 replicas of tablet 660255707c10469a8248ec0520cf875b
I20260808 22:59:55.251915 27466 tablet_service.cc:1514] Processing DeleteTablet for tablet 660255707c10469a8248ec0520cf875b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:55 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:58374
I20260808 22:59:55.251955 27694 tablet_service.cc:1514] Processing DeleteTablet for tablet 660255707c10469a8248ec0520cf875b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:55 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:48214
I20260808 22:59:55.252059 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 27139
I20260808 22:59:55.252156 27244 tablet_service.cc:1514] Processing DeleteTablet for tablet 660255707c10469a8248ec0520cf875b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:59:55 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:38196
I20260808 22:59:55.257576 28045 tablet_replica.cc:330] T 660255707c10469a8248ec0520cf875b P dafd6b7d77a444b6aaa6deb58300d9eb: stopping tablet replica
I20260808 22:59:55.257575 28047 tablet_replica.cc:330] T 660255707c10469a8248ec0520cf875b P a955e828f15f4966a96f217dbecdb2ac: stopping tablet replica
I20260808 22:59:55.257644 28045 raft_consensus.cc:2229] T 660255707c10469a8248ec0520cf875b P dafd6b7d77a444b6aaa6deb58300d9eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:59:55.257644 28047 raft_consensus.cc:2229] T 660255707c10469a8248ec0520cf875b P a955e828f15f4966a96f217dbecdb2ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:59:55.257686 28045 raft_consensus.cc:2258] T 660255707c10469a8248ec0520cf875b P dafd6b7d77a444b6aaa6deb58300d9eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:55.257686 28047 raft_consensus.cc:2258] T 660255707c10469a8248ec0520cf875b P a955e828f15f4966a96f217dbecdb2ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:59:55.257922 28045 ts_tablet_manager.cc:1905] T 660255707c10469a8248ec0520cf875b P dafd6b7d77a444b6aaa6deb58300d9eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:59:55.257930 28047 ts_tablet_manager.cc:1905] T 660255707c10469a8248ec0520cf875b P a955e828f15f4966a96f217dbecdb2ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:59:55.259477 28045 ts_tablet_manager.cc:1918] T 660255707c10469a8248ec0520cf875b P dafd6b7d77a444b6aaa6deb58300d9eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 22:59:55.259532 28045 log.cc:1196] T 660255707c10469a8248ec0520cf875b P dafd6b7d77a444b6aaa6deb58300d9eb: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/660255707c10469a8248ec0520cf875b
I20260808 22:59:55.259734 28045 ts_tablet_manager.cc:1939] T 660255707c10469a8248ec0520cf875b P dafd6b7d77a444b6aaa6deb58300d9eb: Deleting consensus metadata
I20260808 22:59:55.259930 28047 ts_tablet_manager.cc:1918] T 660255707c10469a8248ec0520cf875b P a955e828f15f4966a96f217dbecdb2ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 22:59:55.260010 28047 log.cc:1196] T 660255707c10469a8248ec0520cf875b P a955e828f15f4966a96f217dbecdb2ac: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/660255707c10469a8248ec0520cf875b
I20260808 22:59:55.260028 27013 catalog_manager.cc:4841] TS dafd6b7d77a444b6aaa6deb58300d9eb (127.23.194.3:40031): tablet 660255707c10469a8248ec0520cf875b (table test-table [id=17f3d0df58e744a1b0e19e68e3e4976d]) successfully deleted
I20260808 22:59:55.260257 28047 ts_tablet_manager.cc:1939] T 660255707c10469a8248ec0520cf875b P a955e828f15f4966a96f217dbecdb2ac: Deleting consensus metadata
I20260808 22:59:55.260598 27011 catalog_manager.cc:4841] TS a955e828f15f4966a96f217dbecdb2ac (127.23.194.2:40651): tablet 660255707c10469a8248ec0520cf875b (table test-table [id=17f3d0df58e744a1b0e19e68e3e4976d]) successfully deleted
W20260808 22:59:55.265769 27011 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.194.1:39133 (error 108)
I20260808 22:59:55.265980 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 27367
I20260808 22:59:55.267831 28053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:55.267576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:39133 (local address 127.0.0.1:45646)
0808 22:59:55.267741 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:55.267793 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.1:39133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":104,"threads_started":1}
W20260808 22:59:55.267935 27011 catalog_manager.cc:4584] TS 08e21cbce6254c83bcddee69082c86fe (127.23.194.1:39133): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 660255707c10469a8248ec0520cf875b: Network error: Client connection negotiation failed: client connection to 127.23.194.1:39133: connect: Connection refused (error 111)
I20260808 22:59:55.278765 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 27588
I20260808 22:59:55.287005 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 26988
2026-08-08T22:59:55Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6184 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:55 dist-test-slave-f0tn krb5kdc[28056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:55 dist-test-slave-f0tn krb5kdc[28056](info): set up 2 sockets
Aug 08 22:59:55 dist-test-slave-f0tn krb5kdc[28056](info): commencing operation
krb5kdc: starting...
W20260808 22:59:56.435845 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 22:59:56 dist-test-slave-f0tn krb5kdc[28056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 22:59:56.549471 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:40485
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:42003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:40485
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:56.629185 28078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:56.629426 28078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:56.629460 28078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:56.630584 28078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:56.630661 28078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:56.630695 28078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:56.630720 28078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:56.630738 28078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:56.632256 28078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:42003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:40485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:40485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:56.632513 28078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:56.632723 28078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:56.637805 28091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:56.637816 28088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:56.637951 28078 server_base.cc:1034] running on GCE node
W20260808 22:59:56.637821 28087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:56.638223 28078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:56.638548 28078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:56.639703 28078 hybrid_clock.cc:648] HybridClock initialized: now 1786229996639685 us; error 38 us; skew 500 ppm
Aug 08 22:59:56 dist-test-slave-f0tn krb5kdc[28056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229996, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:56.642053 28078 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 22:59:56.642438 28078 webserver.cc:460] Webserver started at http://127.23.194.62:34395/ using document root <none> and password file <none>
I20260808 22:59:56.642632 28078 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:56.642680 28078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:56.642772 28078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:56.643797 28078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "a94a1d56db16404096b5e8e0d0c40405"
format_stamp: "Formatted at 2026-08-08 22:59:56 on dist-test-slave-f0tn"
server_key: "63cf28a8d47869b43a3707da42b19103"
server_key_iv: "652d9ff2f0032716718e6482fa303e0a"
server_key_version: "encryptionkey@0"
I20260808 22:59:56.644093 28078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "a94a1d56db16404096b5e8e0d0c40405"
format_stamp: "Formatted at 2026-08-08 22:59:56 on dist-test-slave-f0tn"
server_key: "63cf28a8d47869b43a3707da42b19103"
server_key_iv: "652d9ff2f0032716718e6482fa303e0a"
server_key_version: "encryptionkey@0"
I20260808 22:59:56.645233 28078 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:59:56.645942 28104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:56.646088 28078 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:56.646188 28078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "a94a1d56db16404096b5e8e0d0c40405"
format_stamp: "Formatted at 2026-08-08 22:59:56 on dist-test-slave-f0tn"
server_key: "63cf28a8d47869b43a3707da42b19103"
server_key_iv: "652d9ff2f0032716718e6482fa303e0a"
server_key_version: "encryptionkey@0"
I20260808 22:59:56.646247 28078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:56.654517 28078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:56.656131 28078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:56.656243 28078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:56.660071 28078 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:40485
I20260808 22:59:56.660107 28176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:40485 every 8 connection(s)
I20260808 22:59:56.660440 28078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:56.661305 28178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:56.663544 28178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405: Bootstrap starting.
I20260808 22:59:56.663607 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 28078
I20260808 22:59:56.663692 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 22:59:56.663861 24328 external_mini_cluster.cc:1428] Setting key 49e50282fe52439e101d2df0689bbb29
I20260808 22:59:56.664125 28178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:56.664461 28178 log.cc:824] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:56.665102 28178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405: No bootstrap required, opened a new log
I20260808 22:59:56.666899 28178 raft_consensus.cc:348] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94a1d56db16404096b5e8e0d0c40405" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40485 } }
I20260808 22:59:56.667032 28178 raft_consensus.cc:374] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:56.667064 28178 raft_consensus.cc:729] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a94a1d56db16404096b5e8e0d0c40405, State: Initialized, Role: FOLLOWER
I20260808 22:59:56.667189 28178 consensus_queue.cc:260] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94a1d56db16404096b5e8e0d0c40405" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40485 } }
I20260808 22:59:56.667268 28178 raft_consensus.cc:388] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:56.667297 28178 raft_consensus.cc:482] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:56.667344 28178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:56.667841 28178 raft_consensus.cc:504] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94a1d56db16404096b5e8e0d0c40405" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40485 } }
I20260808 22:59:56.667940 28178 leader_election.cc:302] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a94a1d56db16404096b5e8e0d0c40405; no voters:
I20260808 22:59:56.668092 28178 leader_election.cc:288] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:56.668125 28184 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:56.668243 28184 raft_consensus.cc:686] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405 [term 1 LEADER]: Becoming Leader. State: Replica: a94a1d56db16404096b5e8e0d0c40405, State: Running, Role: LEADER
I20260808 22:59:56.668324 28184 consensus_queue.cc:237] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94a1d56db16404096b5e8e0d0c40405" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40485 } }
I20260808 22:59:56.668354 28178 sys_catalog.cc:564] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:56.668851 28186 sys_catalog.cc:455] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a94a1d56db16404096b5e8e0d0c40405. Latest consensus state: current_term: 1 leader_uuid: "a94a1d56db16404096b5e8e0d0c40405" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94a1d56db16404096b5e8e0d0c40405" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40485 } } }
I20260808 22:59:56.669086 28186 sys_catalog.cc:458] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:56.669023 28185 sys_catalog.cc:455] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a94a1d56db16404096b5e8e0d0c40405" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94a1d56db16404096b5e8e0d0c40405" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40485 } } }
I20260808 22:59:56.669370 28185 sys_catalog.cc:458] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:56.669559 28197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:56.670037 28197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 22:59:56 dist-test-slave-f0tn krb5kdc[28056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:56.671672 28197 catalog_manager.cc:1347] Generated new cluster ID: ae1b3ecd3df14353a8a56f24e35ac1e5
I20260808 22:59:56.671728 28197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:59:56.674213 28182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:56.664312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43874 (local address 127.23.194.62:40485)
0808 22:59:56.664505 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:56.664515 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:56.664550 (+ 35us) server_negotiation.cc:407] Connection header received
0808 22:59:56.665059 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:56.665066 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:56.665115 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:56.665273 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:56.665913 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:56.666778 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:56.667649 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:56.668098 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:56.668510 (+ 412us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:56.671055 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:56.671059 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:56.671059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:56.671071 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:56.672730 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:56.673041 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:56.673044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:56.673044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:56.673136 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:56.673310 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:56.673311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:56.673311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:56.673619 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:56.673778 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:56.673873 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 22:59:56.674003 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":87,"threads_started":1}
I20260808 22:59:56.684638 28197 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:56.685134 28197 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:56.688877 28197 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a94a1d56db16404096b5e8e0d0c40405: Generated new TSK 0
I20260808 22:59:56.689041 28197 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
I20260808 22:59:56.723759 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:40485
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:42003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:56.825176 28219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:56.825436 28219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:56.825464 28219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:56.826639 28219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:56.826689 28219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:56.826717 28219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 22:59:56.828204 28219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:42003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:40485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:56.828452 28219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:56.828677 28219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:59:56.833829 28219 server_base.cc:1034] running on GCE node
W20260808 22:59:56.833760 28227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:56.833773 28232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:56.833760 28228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:56.834344 28219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:56.834630 28219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:56.835801 28219 hybrid_clock.cc:648] HybridClock initialized: now 1786229996835784 us; error 31 us; skew 500 ppm
Aug 08 22:59:56 dist-test-slave-f0tn krb5kdc[28056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229996, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:56.838603 28219 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 22:59:56.839035 28219 webserver.cc:460] Webserver started at http://127.23.194.1:40509/ using document root <none> and password file <none>
I20260808 22:59:56.839237 28219 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:56.839277 28219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:56.839370 28219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:56.840385 28219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "c529514ce54544b7b9780b125293fb07"
format_stamp: "Formatted at 2026-08-08 22:59:56 on dist-test-slave-f0tn"
server_key: "c0c880c50d0ac5fcbdd50999639b44c0"
server_key_iv: "146040321305a62199d09dcd61fe7e67"
server_key_version: "encryptionkey@0"
I20260808 22:59:56.840687 28219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "c529514ce54544b7b9780b125293fb07"
format_stamp: "Formatted at 2026-08-08 22:59:56 on dist-test-slave-f0tn"
server_key: "c0c880c50d0ac5fcbdd50999639b44c0"
server_key_iv: "146040321305a62199d09dcd61fe7e67"
server_key_version: "encryptionkey@0"
I20260808 22:59:56.842026 28219 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 22:59:56.842705 28243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:56.842911 28219 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 22:59:56.843000 28219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "c529514ce54544b7b9780b125293fb07"
format_stamp: "Formatted at 2026-08-08 22:59:56 on dist-test-slave-f0tn"
server_key: "c0c880c50d0ac5fcbdd50999639b44c0"
server_key_iv: "146040321305a62199d09dcd61fe7e67"
server_key_version: "encryptionkey@0"
I20260808 22:59:56.843075 28219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:56.852938 28219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:56.855124 28219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:56.855261 28219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:56.855468 28219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:59:56.855772 28219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:59:56.855809 28219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:56.855835 28219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:59:56.855863 28219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:56.862052 28219 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:42255
I20260808 22:59:56.862123 28416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:42255 every 8 connection(s)
I20260808 22:59:56.862419 28219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
Aug 08 22:59:56 dist-test-slave-f0tn krb5kdc[28056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229996, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:56.869451 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 28219
I20260808 22:59:56.869532 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 22:59:56.869719 24328 external_mini_cluster.cc:1428] Setting key eae2aaef2720efd697ff23b349b16eea
I20260808 22:59:56.871613 28182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:56.863211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:45773 (local address 127.23.194.62:40485)
0808 22:59:56.863291 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:56.863293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:56.863477 (+ 184us) server_negotiation.cc:407] Connection header received
0808 22:59:56.863945 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:56.863946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:56.863957 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:56.863999 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:56.864936 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:56.865483 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:56.866402 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:56.866602 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:56.866671 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:56.868877 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:56.868879 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:56.868879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:56.868886 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:56.870329 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:56.870776 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:56.870777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:56.870777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:56.870825 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:56.871065 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:56.871065 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:56.871066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:56.871157 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:56.871201 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:56.871486 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 22:59:56.871548 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 22:59:56.871851 28420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:56.863145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40485 (local address 127.23.194.1:45773)
0808 22:59:56.863360 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:56.863400 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:56.863849 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:56.864066 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:56.864070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:56.864642 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:56.864839 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:56.864845 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:56.865551 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:56.865552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:56.866337 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:56.866339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:56.866676 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:56.866677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:56.866771 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:56.867202 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:56.867210 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:56.868803 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:56.870401 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:56.870409 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:56.870411 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:56.870710 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:56.870877 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:56.870877 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:56.870878 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:56.871012 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:56.871209 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:56.871214 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:56.871364 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 22:59:56.871476 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 22:59:56.871629 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":74,"threads_started":1}
I20260808 22:59:56.872364 28417 heartbeater.cc:347] Connected to a master server at 127.23.194.62:40485
I20260808 22:59:56.872467 28417 heartbeater.cc:464] Registering TS with master...
I20260808 22:59:56.872654 28417 heartbeater.cc:511] Master 127.23.194.62:40485 requested a full tablet report, sending...
I20260808 22:59:56.873158 28132 ts_manager.cc:194] Registered new tserver with Master: c529514ce54544b7b9780b125293fb07 (127.23.194.1:42255)
I20260808 22:59:56.873813 28132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:45773
WARNING: no policy specified for kudu/127.23.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
I20260808 22:59:56.892601 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:40485
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:42003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:56.973259 28425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:56.973456 28425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:56.973479 28425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:56.974599 28425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:56.974639 28425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:56.974674 28425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 22:59:56.976126 28425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:42003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:40485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:56.976359 28425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:56.976580 28425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:56.981600 28434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:56.981911 28430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:56.981738 28432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:56.981809 28425 server_base.cc:1034] running on GCE node
I20260808 22:59:56.982321 28425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:56.982556 28425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:56.983700 28425 hybrid_clock.cc:648] HybridClock initialized: now 1786229996983683 us; error 34 us; skew 500 ppm
Aug 08 22:59:56 dist-test-slave-f0tn krb5kdc[28056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229996, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:56.986230 28425 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 22:59:56.986660 28425 webserver.cc:460] Webserver started at http://127.23.194.2:34323/ using document root <none> and password file <none>
I20260808 22:59:56.986891 28425 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:56.986938 28425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:56.987082 28425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:56.988086 28425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "def7c54c5d8841ed83221563d71e7e42"
format_stamp: "Formatted at 2026-08-08 22:59:56 on dist-test-slave-f0tn"
server_key: "6f2675d4d5d0031765daa62daa7bb4ba"
server_key_iv: "a52d6f64d69bab5107ee4289e2a1daad"
server_key_version: "encryptionkey@0"
I20260808 22:59:56.988423 28425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "def7c54c5d8841ed83221563d71e7e42"
format_stamp: "Formatted at 2026-08-08 22:59:56 on dist-test-slave-f0tn"
server_key: "6f2675d4d5d0031765daa62daa7bb4ba"
server_key_iv: "a52d6f64d69bab5107ee4289e2a1daad"
server_key_version: "encryptionkey@0"
I20260808 22:59:56.989751 28425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 22:59:56.990439 28443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:56.990584 28425 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 22:59:56.990662 28425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "def7c54c5d8841ed83221563d71e7e42"
format_stamp: "Formatted at 2026-08-08 22:59:56 on dist-test-slave-f0tn"
server_key: "6f2675d4d5d0031765daa62daa7bb4ba"
server_key_iv: "a52d6f64d69bab5107ee4289e2a1daad"
server_key_version: "encryptionkey@0"
I20260808 22:59:56.990729 28425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:57.004014 28425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:57.005692 28425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:57.005813 28425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:57.006017 28425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:59:57.006332 28425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:59:57.006366 28425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:57.006388 28425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:59:57.006412 28425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:57.011931 28425 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:35611
I20260808 22:59:57.011976 28612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:35611 every 8 connection(s)
I20260808 22:59:57.012257 28425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 22:59:57.016974 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 28425
I20260808 22:59:57.017073 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 22:59:57.017279 24328 external_mini_cluster.cc:1428] Setting key 450c5ffefffa293d4ff08c0780519e90
Aug 08 22:59:57 dist-test-slave-f0tn krb5kdc[28056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229996, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:57.021328 28182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:57.012902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:44075 (local address 127.23.194.62:40485)
0808 22:59:57.012964 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:57.012967 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:57.013144 (+ 177us) server_negotiation.cc:407] Connection header received
0808 22:59:57.013697 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:57.013698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:57.013709 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:57.013758 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:57.014656 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:57.015132 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:57.016076 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:57.016256 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:57.016301 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:57.018535 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:57.018537 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:57.018537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:57.018545 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:57.019931 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:57.020391 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:57.020392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:57.020392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:57.020443 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:57.020701 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:57.020702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:57.020702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:57.020798 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:57.020844 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:57.021145 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 22:59:57.021242 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 22:59:57.021412 28615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:57.012872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40485 (local address 127.23.194.2:44075)
0808 22:59:57.013030 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:57.013065 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:57.013569 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:57.013838 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:57.013842 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:57.014397 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:57.014577 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:57.014582 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:57.015218 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:57.015219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:57.016014 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:57.016017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:57.016313 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:57.016314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:57.016414 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:57.016846 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:57.016856 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:57.018433 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:57.020011 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:57.020018 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:57.020020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:57.020315 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:57.020506 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:57.020507 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:57.020507 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:57.020639 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:57.020855 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:57.020860 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:57.021003 (+ 143us) client_negotiation.cc:769] Sending connection context
0808 22:59:57.021105 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 22:59:57.021228 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260808 22:59:57.021914 28613 heartbeater.cc:347] Connected to a master server at 127.23.194.62:40485
I20260808 22:59:57.022017 28613 heartbeater.cc:464] Registering TS with master...
I20260808 22:59:57.022192 28613 heartbeater.cc:511] Master 127.23.194.62:40485 requested a full tablet report, sending...
I20260808 22:59:57.022603 28132 ts_manager.cc:194] Registered new tserver with Master: def7c54c5d8841ed83221563d71e7e42 (127.23.194.2:35611)
I20260808 22:59:57.023108 28132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:44075
WARNING: no policy specified for kudu/127.23.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
I20260808 22:59:57.041296 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:40485
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:42003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:57.125447 28621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:57.125649 28621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:57.125680 28621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:57.126847 28621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:57.126888 28621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:57.126924 28621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 22:59:57.128336 28621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:42003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:40485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:57.128561 28621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:57.128767 28621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:57.133675 28630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:57.133754 28627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:57.133713 28632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:57.133777 28621 server_base.cc:1034] running on GCE node
I20260808 22:59:57.134059 28621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:57.134307 28621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:57.135475 28621 hybrid_clock.cc:648] HybridClock initialized: now 1786229997135455 us; error 31 us; skew 500 ppm
Aug 08 22:59:57 dist-test-slave-f0tn krb5kdc[28056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229997, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:57.138039 28621 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 22:59:57.138464 28621 webserver.cc:460] Webserver started at http://127.23.194.3:42089/ using document root <none> and password file <none>
I20260808 22:59:57.138662 28621 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:57.138710 28621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:57.138834 28621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:57.139842 28621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "9d3063aedddb4ea18e813137be0544b4"
format_stamp: "Formatted at 2026-08-08 22:59:57 on dist-test-slave-f0tn"
server_key: "816d841a45102f274463815194665c90"
server_key_iv: "549fcab629297b8ff997ea9a38b56553"
server_key_version: "encryptionkey@0"
I20260808 22:59:57.140182 28621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "9d3063aedddb4ea18e813137be0544b4"
format_stamp: "Formatted at 2026-08-08 22:59:57 on dist-test-slave-f0tn"
server_key: "816d841a45102f274463815194665c90"
server_key_iv: "549fcab629297b8ff997ea9a38b56553"
server_key_version: "encryptionkey@0"
I20260808 22:59:57.141481 28621 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:59:57.142288 28643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:57.142443 28621 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:57.142529 28621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "9d3063aedddb4ea18e813137be0544b4"
format_stamp: "Formatted at 2026-08-08 22:59:57 on dist-test-slave-f0tn"
server_key: "816d841a45102f274463815194665c90"
server_key_iv: "549fcab629297b8ff997ea9a38b56553"
server_key_version: "encryptionkey@0"
I20260808 22:59:57.142597 28621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:57.149569 28621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:57.151434 28621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:57.151540 28621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:57.151715 28621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:59:57.151959 28621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:59:57.151988 28621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:57.152006 28621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:59:57.152019 28621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:57.157637 28621 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:44825
I20260808 22:59:57.157709 28846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:44825 every 8 connection(s)
I20260808 22:59:57.157956 28621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
Aug 08 22:59:57 dist-test-slave-f0tn krb5kdc[28056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229997, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:57.165992 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 28621
I20260808 22:59:57.166107 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 22:59:57.166329 24328 external_mini_cluster.cc:1428] Setting key ab47ae306f3a050d6e49ab7bbe4c76ba
I20260808 22:59:57.166873 28182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:57.158591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:36599 (local address 127.23.194.62:40485)
0808 22:59:57.158664 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:57.158666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:57.158829 (+ 163us) server_negotiation.cc:407] Connection header received
0808 22:59:57.159293 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:57.159294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:57.159304 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:57.159352 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:57.160201 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:57.160689 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:57.161689 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:57.161889 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:57.161939 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:57.164091 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:57.164093 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:57.164093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:57.164100 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:57.165423 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:57.165819 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:57.165820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:57.165820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:57.165872 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:57.166217 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:57.166217 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:57.166218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:57.166331 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:57.166391 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:57.166700 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 22:59:57.166785 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:59:57.167094 28849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:57.158565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40485 (local address 127.23.194.3:36599)
0808 22:59:57.158721 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 22:59:57.158755 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 22:59:57.159217 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:59:57.159403 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:59:57.159406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:59:57.159938 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:59:57.160123 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:57.160127 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:57.160749 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:57.160749 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:57.161607 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:59:57.161610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:59:57.161960 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:59:57.161961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:59:57.162082 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:57.162522 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:59:57.162531 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:59:57.163951 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:59:57.165500 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:57.165507 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:57.165509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:57.165737 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:57.165931 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:59:57.165932 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:59:57.165932 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:59:57.166109 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:59:57.166397 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:59:57.166404 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:59:57.166555 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 22:59:57.166668 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 22:59:57.166856 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":96,"threads_started":1}
I20260808 22:59:57.167515 28847 heartbeater.cc:347] Connected to a master server at 127.23.194.62:40485
I20260808 22:59:57.167609 28847 heartbeater.cc:464] Registering TS with master...
I20260808 22:59:57.167816 28847 heartbeater.cc:511] Master 127.23.194.62:40485 requested a full tablet report, sending...
I20260808 22:59:57.168188 28132 ts_manager.cc:194] Registered new tserver with Master: 9d3063aedddb4ea18e813137be0544b4 (127.23.194.3:44825)
I20260808 22:59:57.168565 28132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:36599
I20260808 22:59:57.178193 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 22:59:57.191314 28182 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:59:57.190383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43884 (local address 127.23.194.62:40485)
0808 22:59:57.190443 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:57.190445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:57.190563 (+ 118us) server_negotiation.cc:407] Connection header received
0808 22:59:57.190614 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:57.190615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:57.190626 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:57.190688 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:57.191241 (+ 553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43884: BlockingRecv error: recv got EOF from 127.0.0.1:43884 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 22:59:57.192178 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 28219
I20260808 22:59:57.208817 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 28425
I20260808 22:59:57.224459 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 28621
I20260808 22:59:57.232172 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 28078
2026-08-08T22:59:57Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1944 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:57 dist-test-slave-f0tn krb5kdc[28861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:57 dist-test-slave-f0tn krb5kdc[28861](info): set up 2 sockets
Aug 08 22:59:57 dist-test-slave-f0tn krb5kdc[28861](info): commencing operation
krb5kdc: starting...
W20260808 22:59:58.376199 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 22:59:58 dist-test-slave-f0tn krb5kdc[28861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 22:59:58.487761 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:42943
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:34483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:42943
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:58.570870 28882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:58.571075 28882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:58.571110 28882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:58.572220 28882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:58.572260 28882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:58.572278 28882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:58.572294 28882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:58.572311 28882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:58.573822 28882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:34483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:42943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:42943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:58.574064 28882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:58.574259 28882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:58.578954 28895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:58.579015 28900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:58.578987 28896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:58.579202 28882 server_base.cc:1034] running on GCE node
I20260808 22:59:58.579434 28882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:58.579723 28882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:58.580861 28882 hybrid_clock.cc:648] HybridClock initialized: now 1786229998580849 us; error 28 us; skew 500 ppm
Aug 08 22:59:58 dist-test-slave-f0tn krb5kdc[28861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229998, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:58.587198 28882 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 22:59:58.587618 28882 webserver.cc:460] Webserver started at http://127.23.194.62:34963/ using document root <none> and password file <none>
I20260808 22:59:58.587800 28882 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:58.587839 28882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:58.587931 28882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:58.588912 28882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "be08404a049e4e57bfc045f747877a99"
format_stamp: "Formatted at 2026-08-08 22:59:58 on dist-test-slave-f0tn"
server_key: "b4a0e82dbe25997e975d7e938bb8e42a"
server_key_iv: "83862b90888a398acd10ccf56f8d2522"
server_key_version: "encryptionkey@0"
I20260808 22:59:58.589212 28882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "be08404a049e4e57bfc045f747877a99"
format_stamp: "Formatted at 2026-08-08 22:59:58 on dist-test-slave-f0tn"
server_key: "b4a0e82dbe25997e975d7e938bb8e42a"
server_key_iv: "83862b90888a398acd10ccf56f8d2522"
server_key_version: "encryptionkey@0"
I20260808 22:59:58.590448 28882 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 22:59:58.591084 28911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:59:58.591238 28882 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 22:59:58.591322 28882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "be08404a049e4e57bfc045f747877a99"
format_stamp: "Formatted at 2026-08-08 22:59:58 on dist-test-slave-f0tn"
server_key: "b4a0e82dbe25997e975d7e938bb8e42a"
server_key_iv: "83862b90888a398acd10ccf56f8d2522"
server_key_version: "encryptionkey@0"
I20260808 22:59:58.591377 28882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:59:58.616549 28882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:59:58.618284 28882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:59:58.618394 28882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:59:58.622040 28882 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:42943
I20260808 22:59:58.622087 28987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:42943 every 8 connection(s)
I20260808 22:59:58.622611 28882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 22:59:58.623270 28990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:59:58.625554 28990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99: Bootstrap starting.
I20260808 22:59:58.626132 28990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99: Neither blocks nor log segments found. Creating new log.
I20260808 22:59:58.626418 28990 log.cc:824] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99: Log is configured to *not* fsync() on all Append() calls
I20260808 22:59:58.627058 28990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99: No bootstrap required, opened a new log
I20260808 22:59:58.628620 28990 raft_consensus.cc:348] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be08404a049e4e57bfc045f747877a99" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42943 } }
I20260808 22:59:58.628762 28990 raft_consensus.cc:374] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:59:58.628795 28990 raft_consensus.cc:729] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be08404a049e4e57bfc045f747877a99, State: Initialized, Role: FOLLOWER
I20260808 22:59:58.628942 28990 consensus_queue.cc:260] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be08404a049e4e57bfc045f747877a99" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42943 } }
I20260808 22:59:58.629004 28990 raft_consensus.cc:388] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:59:58.629025 28990 raft_consensus.cc:482] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:59:58.629072 28990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:59:58.629987 28990 raft_consensus.cc:504] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be08404a049e4e57bfc045f747877a99" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42943 } }
I20260808 22:59:58.630082 28990 leader_election.cc:302] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be08404a049e4e57bfc045f747877a99; no voters:
I20260808 22:59:58.630249 28990 leader_election.cc:288] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:59:58.630286 28993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:59:58.630411 28993 raft_consensus.cc:686] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99 [term 1 LEADER]: Becoming Leader. State: Replica: be08404a049e4e57bfc045f747877a99, State: Running, Role: LEADER
I20260808 22:59:58.630499 28990 sys_catalog.cc:564] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:59:58.630510 28993 consensus_queue.cc:237] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be08404a049e4e57bfc045f747877a99" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42943 } }
I20260808 22:59:58.630893 28995 sys_catalog.cc:455] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be08404a049e4e57bfc045f747877a99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be08404a049e4e57bfc045f747877a99" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42943 } } }
I20260808 22:59:58.630993 28995 sys_catalog.cc:458] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:58.630890 28996 sys_catalog.cc:455] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be08404a049e4e57bfc045f747877a99. Latest consensus state: current_term: 1 leader_uuid: "be08404a049e4e57bfc045f747877a99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be08404a049e4e57bfc045f747877a99" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42943 } } }
I20260808 22:59:58.631155 28996 sys_catalog.cc:458] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99 [sys.catalog]: This master's current role is: LEADER
I20260808 22:59:58.631201 29004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:59:58.631771 29004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:59:58.632624 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 28882
I20260808 22:59:58.632735 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 22:59:58.632920 24328 external_mini_cluster.cc:1428] Setting key 9e8ac207940fb354bd7754b9a192ce00
I20260808 22:59:58.633317 29004 catalog_manager.cc:1347] Generated new cluster ID: f46960f7455b4a38afc8ff544e5fcfc4
I20260808 22:59:58.633369 29004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 22:59:58 dist-test-slave-f0tn krb5kdc[28861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 22:59:58.641894 29004 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:59:58.642370 29004 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:59:58.644333 29023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:58.633531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38120 (local address 127.23.194.62:42943)
0808 22:59:58.633754 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:58.633762 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:58.633791 (+ 29us) server_negotiation.cc:407] Connection header received
0808 22:59:58.633902 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:58.633909 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:58.633961 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:58.634080 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:58.634903 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:58.636263 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:58.637327 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:58.637947 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:58.638039 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:58.640754 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:58.640759 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:58.640759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:59:58.640772 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:58.642801 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:58.643170 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:58.643174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:58.643174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:58.643282 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:59:58.643450 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:59:58.643450 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:59:58.643451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:59:58.643719 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:58.643864 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:58.643935 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 22:59:58.644081 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":77,"threads_started":1}
I20260808 22:59:58.645265 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:59:58.651547 29004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be08404a049e4e57bfc045f747877a99: Generated new TSK 0
I20260808 22:59:58.651726 29004 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:59:58.660676 29023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:59:58.657423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38134 (local address 127.23.194.62:42943)
0808 22:59:58.657472 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 22:59:58.657474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 22:59:58.657591 (+ 117us) server_negotiation.cc:407] Connection header received
0808 22:59:58.657661 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:59:58.657662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:59:58.657673 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:59:58.657719 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:59:58.658077 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:58.658582 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:58.659299 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:59:58.659528 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:59:58.659580 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:59:58.660251 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:59:58.660253 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:59:58.660253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:59:58.660431 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:59:58.660447 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:59:58.660452 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:59:58.660499 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 22:59:58.660566 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 22:59:58.660592 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 22:59:58.662985 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 28882
2026-08-08T22:59:58Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1430 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:59:58 dist-test-slave-f0tn krb5kdc[29036](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:59:58 dist-test-slave-f0tn krb5kdc[29036](info): set up 2 sockets
Aug 08 22:59:58 dist-test-slave-f0tn krb5kdc[29036](info): commencing operation
krb5kdc: starting...
W20260808 22:59:59.789587 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 22:59:59 dist-test-slave-f0tn krb5kdc[29036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:59:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:59:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 22:59:59.902210 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:35441
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:43845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:35441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:59:59.983021 29054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:59:59.983215 29054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:59:59.983237 29054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:59:59.984371 29054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:59:59.984412 29054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:59:59.984426 29054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:59:59.984443 29054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:59:59.984467 29054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:59:59.985945 29054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:43845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:35441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:35441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 22:59:59.986172 29054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:59:59.986367 29054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:59:59.991086 29066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:59:59.991094 29061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:59.991353 29054 server_base.cc:1034] running on GCE node
W20260808 22:59:59.991104 29062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:59:59.991750 29054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:59:59.992100 29054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:59:59.993217 29054 hybrid_clock.cc:648] HybridClock initialized: now 1786229999993193 us; error 40 us; skew 500 ppm
Aug 08 22:59:59 dist-test-slave-f0tn krb5kdc[29036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229999, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:59:59.995879 29054 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 22:59:59.996306 29054 webserver.cc:460] Webserver started at http://127.23.194.62:36899/ using document root <none> and password file <none>
I20260808 22:59:59.996541 29054 fs_manager.cc:359] Metadata directory not provided
I20260808 22:59:59.996587 29054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:59:59.996716 29054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:59:59.997786 29054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "b010b5fe89174d2a9900626ab9ceff1f"
format_stamp: "Formatted at 2026-08-08 22:59:59 on dist-test-slave-f0tn"
server_key: "337ea31e262e2418c186809871383339"
server_key_iv: "7f0412da689039569c2caefc774e799b"
server_key_version: "encryptionkey@0"
I20260808 22:59:59.998111 29054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "b010b5fe89174d2a9900626ab9ceff1f"
format_stamp: "Formatted at 2026-08-08 22:59:59 on dist-test-slave-f0tn"
server_key: "337ea31e262e2418c186809871383339"
server_key_iv: "7f0412da689039569c2caefc774e799b"
server_key_version: "encryptionkey@0"
I20260808 22:59:59.999323 29054 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:59:59.999987 29079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:00.000136 29054 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:00.000206 29054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "b010b5fe89174d2a9900626ab9ceff1f"
format_stamp: "Formatted at 2026-08-08 22:59:59 on dist-test-slave-f0tn"
server_key: "337ea31e262e2418c186809871383339"
server_key_iv: "7f0412da689039569c2caefc774e799b"
server_key_version: "encryptionkey@0"
I20260808 23:00:00.000272 29054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:00.016863 29054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:00.018842 29054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:00.018958 29054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:00.022791 29054 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:35441
I20260808 23:00:00.022830 29176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:35441 every 8 connection(s)
I20260808 23:00:00.023157 29054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:00:00.024371 29177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:00.026628 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 29054
I20260808 23:00:00.026713 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:00:00.026854 29177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f: Bootstrap starting.
I20260808 23:00:00.026880 24328 external_mini_cluster.cc:1428] Setting key 195489340c040e32ebacaab25b121913
I20260808 23:00:00.027552 29177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:00.027865 29177 log.cc:824] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:00.028605 29177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f: No bootstrap required, opened a new log
I20260808 23:00:00.030895 29177 raft_consensus.cc:348] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b010b5fe89174d2a9900626ab9ceff1f" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35441 } }
I20260808 23:00:00.031037 29177 raft_consensus.cc:374] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:00.031060 29177 raft_consensus.cc:729] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b010b5fe89174d2a9900626ab9ceff1f, State: Initialized, Role: FOLLOWER
I20260808 23:00:00.031172 29177 consensus_queue.cc:260] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b010b5fe89174d2a9900626ab9ceff1f" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35441 } }
I20260808 23:00:00.031256 29177 raft_consensus.cc:388] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:00:00.031289 29177 raft_consensus.cc:482] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:00:00.031335 29177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:00.031869 29177 raft_consensus.cc:504] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b010b5fe89174d2a9900626ab9ceff1f" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35441 } }
I20260808 23:00:00.031968 29177 leader_election.cc:302] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b010b5fe89174d2a9900626ab9ceff1f; no voters:
I20260808 23:00:00.032138 29177 leader_election.cc:288] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:00:00.032192 29184 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:00.032387 29177 sys_catalog.cc:564] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:00:00.032418 29184 raft_consensus.cc:686] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f [term 1 LEADER]: Becoming Leader. State: Replica: b010b5fe89174d2a9900626ab9ceff1f, State: Running, Role: LEADER
I20260808 23:00:00.032563 29184 consensus_queue.cc:237] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b010b5fe89174d2a9900626ab9ceff1f" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35441 } }
I20260808 23:00:00.032950 29184 sys_catalog.cc:455] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b010b5fe89174d2a9900626ab9ceff1f. Latest consensus state: current_term: 1 leader_uuid: "b010b5fe89174d2a9900626ab9ceff1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b010b5fe89174d2a9900626ab9ceff1f" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35441 } } }
I20260808 23:00:00.033046 29184 sys_catalog.cc:458] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:00.033617 29205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:00:00.033658 29187 sys_catalog.cc:455] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b010b5fe89174d2a9900626ab9ceff1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b010b5fe89174d2a9900626ab9ceff1f" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35441 } } }
I20260808 23:00:00.033722 29187 sys_catalog.cc:458] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:00.034207 29205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 23:00:00 dist-test-slave-f0tn krb5kdc[29036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786229999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:00.035962 29205 catalog_manager.cc:1347] Generated new cluster ID: 1355b08d3527486188f3cb2c48d547a1
I20260808 23:00:00.036015 29205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:00:00.038412 29181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:00.027425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33834 (local address 127.23.194.62:35441)
0808 23:00:00.027597 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:00.027606 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:00.027642 (+ 36us) server_negotiation.cc:407] Connection header received
0808 23:00:00.028094 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:00.028100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:00.028149 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:00.028269 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:00.028972 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:00.030362 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:00.031218 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:00.031694 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:00.031797 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:00.034825 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:00.034830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:00.034830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:00.034843 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:00.036774 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:00.037191 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:00.037194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:00.037194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:00.037287 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:00.037532 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:00.037533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:00.037533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:00.037808 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:00.037947 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:00.038084 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 23:00:00.038209 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":72,"threads_started":1}
I20260808 23:00:00.064590 29205 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:00:00.065138 29205 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:00:00.069397 29205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b010b5fe89174d2a9900626ab9ceff1f: Generated new TSK 0
I20260808 23:00:00.069582 29205 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
I20260808 23:00:00.101320 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35441
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:43845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:00.202963 29225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:00.203158 29225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:00.203181 29225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:00.204309 29225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:00.204348 29225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:00.204373 29225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:00:00.206219 29225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:43845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:00.206472 29225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:00.206707 29225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:00.211580 29233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:00.211578 29236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:00.211585 29234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:00.211874 29225 server_base.cc:1034] running on GCE node
I20260808 23:00:00.212062 29225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:00.212291 29225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:00.213447 29225 hybrid_clock.cc:648] HybridClock initialized: now 1786230000213433 us; error 28 us; skew 500 ppm
Aug 08 23:00:00 dist-test-slave-f0tn krb5kdc[29036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230000, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:00.215894 29225 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 23:00:00.216337 29225 webserver.cc:460] Webserver started at http://127.23.194.1:44307/ using document root <none> and password file <none>
I20260808 23:00:00.216536 29225 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:00.216578 29225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:00.216707 29225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:00.217832 29225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "497148222828470392ed7b7c42fd2816"
format_stamp: "Formatted at 2026-08-08 23:00:00 on dist-test-slave-f0tn"
server_key: "aab955425ba154fce47f5575f29f47a2"
server_key_iv: "f21149b96a02e7cedfe7a7d5f68783ab"
server_key_version: "encryptionkey@0"
I20260808 23:00:00.218163 29225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "497148222828470392ed7b7c42fd2816"
format_stamp: "Formatted at 2026-08-08 23:00:00 on dist-test-slave-f0tn"
server_key: "aab955425ba154fce47f5575f29f47a2"
server_key_iv: "f21149b96a02e7cedfe7a7d5f68783ab"
server_key_version: "encryptionkey@0"
I20260808 23:00:00.219426 29225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:00:00.220120 29245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:00.220288 29225 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:00.220368 29225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "497148222828470392ed7b7c42fd2816"
format_stamp: "Formatted at 2026-08-08 23:00:00 on dist-test-slave-f0tn"
server_key: "aab955425ba154fce47f5575f29f47a2"
server_key_iv: "f21149b96a02e7cedfe7a7d5f68783ab"
server_key_version: "encryptionkey@0"
I20260808 23:00:00.220433 29225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:00.245747 29225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:00.247874 29225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:00.248014 29225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:00.248229 29225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:00.248530 29225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:00.248564 29225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:00.248584 29225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:00.248608 29225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:00.254166 29225 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:35977
I20260808 23:00:00.254227 29414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:35977 every 8 connection(s)
I20260808 23:00:00.254525 29225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:00:00.257185 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 29225
I20260808 23:00:00.257278 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:00:00.257500 24328 external_mini_cluster.cc:1428] Setting key 80937f68718b7ed6ce557f5fd8b56d88
Aug 08 23:00:00 dist-test-slave-f0tn krb5kdc[29036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230000, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:00.263490 29181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:00.255182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:44159 (local address 127.23.194.62:35441)
0808 23:00:00.255236 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:00.255238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:00.255428 (+ 190us) server_negotiation.cc:407] Connection header received
0808 23:00:00.255996 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:00.255998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:00.256011 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:00.256071 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:00.257000 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:00.257654 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:00.258616 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:00.258843 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:00.258899 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:00.260861 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:00.260862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:00.260862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:00.260871 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:00.262236 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:00.262603 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:00.262604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:00.262604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:00.262664 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:00.262878 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:00.262879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:00.262879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:00.262996 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:00.263051 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:00.263309 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 23:00:00.263399 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 23:00:00.263652 29417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:00.255167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35441 (local address 127.23.194.1:44159)
0808 23:00:00.255313 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:00.255350 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:00.255895 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:00.256139 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:00.256143 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:00.256708 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:00.256903 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:00.256907 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:00.257739 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:00.257740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:00.258534 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:00.258536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:00.258914 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:00.258915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:00.259043 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:00.259490 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:00.259500 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:00.260773 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:00.262309 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:00.262317 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:00.262319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:00.262537 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:00.262717 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:00.262718 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:00.262718 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:00.262824 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:00.263051 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:00.263058 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:00.263192 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 23:00:00.263281 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 23:00:00.263434 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260808 23:00:00.264438 29415 heartbeater.cc:347] Connected to a master server at 127.23.194.62:35441
I20260808 23:00:00.264526 29415 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:00.264719 29415 heartbeater.cc:511] Master 127.23.194.62:35441 requested a full tablet report, sending...
I20260808 23:00:00.265296 29109 ts_manager.cc:194] Registered new tserver with Master: 497148222828470392ed7b7c42fd2816 (127.23.194.1:35977)
WARNING: no policy specified for kudu/127.23.194.2@KRBTEST.COM; defaulting to no policy
I20260808 23:00:00.266234 29109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:44159
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
I20260808 23:00:00.282486 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35441
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:43845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:00.363227 29420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:00.363435 29420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:00.363457 29420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:00.364521 29420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:00.364562 29420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:00.364588 29420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:00:00.366043 29420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:43845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:00.366243 29420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:00.366416 29420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:00.371335 29426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:00.371418 29420 server_base.cc:1034] running on GCE node
W20260808 23:00:00.371330 29427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:00.371400 29429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:00.371891 29420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:00.372138 29420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:00.373268 29420 hybrid_clock.cc:648] HybridClock initialized: now 1786230000373261 us; error 28 us; skew 500 ppm
Aug 08 23:00:00 dist-test-slave-f0tn krb5kdc[29036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230000, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:00.375792 29420 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 23:00:00.376230 29420 webserver.cc:460] Webserver started at http://127.23.194.2:45775/ using document root <none> and password file <none>
I20260808 23:00:00.376451 29420 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:00.376494 29420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:00.376632 29420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:00.377681 29420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "a1e4b34baaca49a1a055c8c167dba3af"
format_stamp: "Formatted at 2026-08-08 23:00:00 on dist-test-slave-f0tn"
server_key: "8ce322b760cb11d30d8aa8c1d9e72d4e"
server_key_iv: "7bfb8be3397c65b41f64e6a4efde5321"
server_key_version: "encryptionkey@0"
I20260808 23:00:00.377998 29420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "a1e4b34baaca49a1a055c8c167dba3af"
format_stamp: "Formatted at 2026-08-08 23:00:00 on dist-test-slave-f0tn"
server_key: "8ce322b760cb11d30d8aa8c1d9e72d4e"
server_key_iv: "7bfb8be3397c65b41f64e6a4efde5321"
server_key_version: "encryptionkey@0"
I20260808 23:00:00.379184 29420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:00:00.379833 29446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:00.379973 29420 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:00.380050 29420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "a1e4b34baaca49a1a055c8c167dba3af"
format_stamp: "Formatted at 2026-08-08 23:00:00 on dist-test-slave-f0tn"
server_key: "8ce322b760cb11d30d8aa8c1d9e72d4e"
server_key_iv: "7bfb8be3397c65b41f64e6a4efde5321"
server_key_version: "encryptionkey@0"
I20260808 23:00:00.380115 29420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:00.388887 29420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:00.390740 29420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:00.390849 29420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:00.391047 29420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:00.391326 29420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:00.391357 29420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:00.391386 29420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:00.391409 29420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:00.396972 29420 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:45167
I20260808 23:00:00.397023 29616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:45167 every 8 connection(s)
I20260808 23:00:00.397302 29420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:00:00.397564 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 29420
I20260808 23:00:00.397680 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:00:00.397886 24328 external_mini_cluster.cc:1428] Setting key a6c9089d4ae13bf927a082ebf3cd0764
Aug 08 23:00:00 dist-test-slave-f0tn krb5kdc[29036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230000, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.194.3@KRBTEST.COM; defaulting to no policy
I20260808 23:00:00.406665 29181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:00.398055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:60239 (local address 127.23.194.62:35441)
0808 23:00:00.398105 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:00.398108 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:00.398403 (+ 295us) server_negotiation.cc:407] Connection header received
0808 23:00:00.398864 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:00.398866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:00.398877 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:00.398924 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:00.399867 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:00.400356 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:00.401303 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:00.401547 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:00.401603 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:00.404013 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:00.404015 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:00.404015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:00.404024 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:00.405446 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:00.405812 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:00.405813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:00.405813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:00.405872 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:00.406081 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:00.406082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:00.406082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:00.406215 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:00.406287 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:00.406495 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 23:00:00.406576 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 23:00:00.406739 29621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:00.398045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35441 (local address 127.23.194.2:60239)
0808 23:00:00.398281 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:00.398315 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:00.398770 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:00.399005 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:00.399009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:00.399581 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:00.399777 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:00.399782 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:00.400437 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:00.400438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:00.401223 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:00.401225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:00.401613 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:00.401614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:00.401757 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:00.402263 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:00.402274 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:00.403926 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:00.405518 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:00.405524 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:00.405525 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:00.405740 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:00.405932 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:00.405933 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:00.405933 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:00.406019 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:00.406273 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:00.406276 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:00.406380 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 23:00:00.406454 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 23:00:00.406568 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":91,"threads_started":1}
I20260808 23:00:00.407204 29617 heartbeater.cc:347] Connected to a master server at 127.23.194.62:35441
I20260808 23:00:00.407296 29617 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:00.407462 29617 heartbeater.cc:511] Master 127.23.194.62:35441 requested a full tablet report, sending...
I20260808 23:00:00.407830 29109 ts_manager.cc:194] Registered new tserver with Master: a1e4b34baaca49a1a055c8c167dba3af (127.23.194.2:45167)
I20260808 23:00:00.408210 29109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:60239
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
I20260808 23:00:00.422183 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35441
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:43845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:00.501952 29628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:00.502141 29628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:00.502166 29628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:00.503314 29628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:00.503357 29628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:00.503383 29628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:00:00.504840 29628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:43845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:00.505048 29628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:00.505249 29628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:00.509963 29642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:00.510066 29636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:00.510083 29628 server_base.cc:1034] running on GCE node
W20260808 23:00:00.509964 29638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:00.510355 29628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:00.510569 29628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:00.511708 29628 hybrid_clock.cc:648] HybridClock initialized: now 1786230000511702 us; error 33 us; skew 500 ppm
Aug 08 23:00:00 dist-test-slave-f0tn krb5kdc[29036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230000, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:00.514806 29628 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 23:00:00.515265 29628 webserver.cc:460] Webserver started at http://127.23.194.3:46267/ using document root <none> and password file <none>
I20260808 23:00:00.515530 29628 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:00.515591 29628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:00.515733 29628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:00.517046 29628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "1a6675aa18c546fa87ae13ae570b86eb"
format_stamp: "Formatted at 2026-08-08 23:00:00 on dist-test-slave-f0tn"
server_key: "39a3a9ef52f3a2a28463c2be0ea68af2"
server_key_iv: "cba531312bef7e2b8d5a3db5c9120ebc"
server_key_version: "encryptionkey@0"
I20260808 23:00:00.517488 29628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "1a6675aa18c546fa87ae13ae570b86eb"
format_stamp: "Formatted at 2026-08-08 23:00:00 on dist-test-slave-f0tn"
server_key: "39a3a9ef52f3a2a28463c2be0ea68af2"
server_key_iv: "cba531312bef7e2b8d5a3db5c9120ebc"
server_key_version: "encryptionkey@0"
I20260808 23:00:00.522734 29628 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.000s
I20260808 23:00:00.523614 29651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:00.523766 29628 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:00.523887 29628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "1a6675aa18c546fa87ae13ae570b86eb"
format_stamp: "Formatted at 2026-08-08 23:00:00 on dist-test-slave-f0tn"
server_key: "39a3a9ef52f3a2a28463c2be0ea68af2"
server_key_iv: "cba531312bef7e2b8d5a3db5c9120ebc"
server_key_version: "encryptionkey@0"
I20260808 23:00:00.523965 29628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:00.536000 29628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:00.537997 29628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:00.538123 29628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:00.538311 29628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:00.538579 29628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:00.538611 29628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:00.538648 29628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:00.538672 29628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:00.544291 29628 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:43557
I20260808 23:00:00.544348 29838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:43557 every 8 connection(s)
I20260808 23:00:00.544623 29628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 23:00:00.547123 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 29628
I20260808 23:00:00.547204 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:00:00.547364 24328 external_mini_cluster.cc:1428] Setting key 138983c578d98888ae49e894248ca0d8
Aug 08 23:00:00 dist-test-slave-f0tn krb5kdc[29036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230000, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:00.552932 29181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:00.545278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:34409 (local address 127.23.194.62:35441)
0808 23:00:00.545333 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:00.545336 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:00.545532 (+ 196us) server_negotiation.cc:407] Connection header received
0808 23:00:00.546014 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:00.546016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:00.546028 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:00.546074 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:00.546972 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:00.547593 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:00.548714 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:00.548898 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:00.548947 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:00.550786 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:00.550787 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:00.550787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:00.550794 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:00.551972 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:00.552281 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:00.552281 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:00.552282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:00.552326 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:00.552489 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:00.552489 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:00.552490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:00.552580 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:00.552622 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:00.552815 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 23:00:00.552871 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 23:00:00.553102 29844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:00.545235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35441 (local address 127.23.194.3:34409)
0808 23:00:00.545419 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:00.545459 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:00.545924 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:00.546151 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:00.546155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:00.546697 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:00.546884 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:00.546889 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:00.547699 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:00.547701 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:00.548641 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:00.548644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:00.548954 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:00.548955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:00.549048 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:00.549453 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:00.549463 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:00.550700 (+ 1237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:00.552030 (+ 1330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:00.552032 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:00.552034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:00.552232 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:00.552368 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:00.552368 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:00.552368 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:00.552450 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:00.552620 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:00.552622 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:00.552725 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 23:00:00.552798 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 23:00:00.552906 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
I20260808 23:00:00.553537 29840 heartbeater.cc:347] Connected to a master server at 127.23.194.62:35441
I20260808 23:00:00.553622 29840 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:00.553794 29840 heartbeater.cc:511] Master 127.23.194.62:35441 requested a full tablet report, sending...
I20260808 23:00:00.554112 29109 ts_manager.cc:194] Registered new tserver with Master: 1a6675aa18c546fa87ae13ae570b86eb (127.23.194.3:43557)
I20260808 23:00:00.554456 29109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:34409
I20260808 23:00:00.558697 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 23:00:00 dist-test-slave-f0tn krb5kdc[29036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230000, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 23:00:00 dist-test-slave-f0tn krb5kdc[29036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230000, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:00.581316 29181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:00.574788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33850 (local address 127.23.194.62:35441)
0808 23:00:00.574845 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:00.574848 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:00.574944 (+ 96us) server_negotiation.cc:407] Connection header received
0808 23:00:00.575015 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:00.575016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:00.575027 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:00.575072 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:00.575614 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:00.576112 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:00.576826 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:00.577006 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:00.577058 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:00.579104 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:00.579105 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:00.579105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:00.579113 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:00.580305 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:00.580592 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:00.580593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:00.580593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:00.580651 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:00.580811 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:00.580811 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:00.580812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:00.580953 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:00.581041 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:00.581157 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 23:00:00.581226 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260808 23:00:00.581529 29109 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33850
I20260808 23:00:00.581902 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 29225
I20260808 23:00:00.594622 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 29420
I20260808 23:00:00.606724 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 29628
I20260808 23:00:00.620359 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 29054
2026-08-08T23:00:00Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1956 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:00:00 dist-test-slave-f0tn krb5kdc[29863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:00:00 dist-test-slave-f0tn krb5kdc[29863](info): set up 2 sockets
Aug 08 23:00:00 dist-test-slave-f0tn krb5kdc[29863](info): commencing operation
krb5kdc: starting...
W20260808 23:00:01.759739 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:00:01 dist-test-slave-f0tn krb5kdc[29863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:00:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:00:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:00:01.875735 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:35047
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:43949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:35047
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:01.956903 29885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:01.957101 29885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:01.957121 29885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:01.958277 29885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:00:01.958319 29885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:01.958333 29885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:01.958344 29885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:00:01.958366 29885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:00:01.959833 29885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:43949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:35047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:35047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:01.960065 29885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:01.960269 29885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:01.964990 29903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:01.965020 29899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:01.965003 29898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:01.965263 29885 server_base.cc:1034] running on GCE node
I20260808 23:00:01.965520 29885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:01.965797 29885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:01.966941 29885 hybrid_clock.cc:648] HybridClock initialized: now 1786230001966926 us; error 30 us; skew 500 ppm
Aug 08 23:00:01 dist-test-slave-f0tn krb5kdc[29863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230001, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:01.969301 29885 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:00:01.969734 29885 webserver.cc:460] Webserver started at http://127.23.194.62:44771/ using document root <none> and password file <none>
I20260808 23:00:01.969908 29885 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:01.969946 29885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:01.970036 29885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:01.971055 29885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "64e0031dad294efda5c77af61823296d"
format_stamp: "Formatted at 2026-08-08 23:00:01 on dist-test-slave-f0tn"
server_key: "64c90dacdee54f0077ca3d12f84b608b"
server_key_iv: "bbd9edf3aaea42fce0c201c87d3c0dde"
server_key_version: "encryptionkey@0"
I20260808 23:00:01.971342 29885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "64e0031dad294efda5c77af61823296d"
format_stamp: "Formatted at 2026-08-08 23:00:01 on dist-test-slave-f0tn"
server_key: "64c90dacdee54f0077ca3d12f84b608b"
server_key_iv: "bbd9edf3aaea42fce0c201c87d3c0dde"
server_key_version: "encryptionkey@0"
I20260808 23:00:01.972788 29885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 23:00:01.973459 29912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:01.973614 29885 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:01.973701 29885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "64e0031dad294efda5c77af61823296d"
format_stamp: "Formatted at 2026-08-08 23:00:01 on dist-test-slave-f0tn"
server_key: "64c90dacdee54f0077ca3d12f84b608b"
server_key_iv: "bbd9edf3aaea42fce0c201c87d3c0dde"
server_key_version: "encryptionkey@0"
I20260808 23:00:01.973770 29885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:01.985888 29885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:01.987489 29885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:01.987617 29885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:01.991333 29885 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:35047
I20260808 23:00:01.991370 29995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:35047 every 8 connection(s)
I20260808 23:00:01.991823 29885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:00:01.992462 29996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:01.995000 29996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d: Bootstrap starting.
I20260808 23:00:01.995586 29996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:01.995832 29996 log.cc:824] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:01.996436 29996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d: No bootstrap required, opened a new log
I20260808 23:00:01.998102 29996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e0031dad294efda5c77af61823296d" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35047 } }
I20260808 23:00:01.998271 29996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:01.998307 29996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64e0031dad294efda5c77af61823296d, State: Initialized, Role: FOLLOWER
I20260808 23:00:01.998447 29996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e0031dad294efda5c77af61823296d" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35047 } }
I20260808 23:00:01.998519 29996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:00:01.998586 29996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:00:01.998646 29996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:01.999679 29996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e0031dad294efda5c77af61823296d" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35047 } }
I20260808 23:00:01.999789 29996 leader_election.cc:302] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64e0031dad294efda5c77af61823296d; no voters:
I20260808 23:00:01.999967 29996 leader_election.cc:288] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:00:02.000149 30001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:02.000216 29996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:00:02.000294 30001 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d [term 1 LEADER]: Becoming Leader. State: Replica: 64e0031dad294efda5c77af61823296d, State: Running, Role: LEADER
I20260808 23:00:02.000391 30001 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e0031dad294efda5c77af61823296d" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35047 } }
I20260808 23:00:02.000602 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 29885
I20260808 23:00:02.000713 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:00:02.000916 24328 external_mini_cluster.cc:1428] Setting key 4ee32786f4cf652a5de01738d2614aa1
I20260808 23:00:02.000988 30001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64e0031dad294efda5c77af61823296d. Latest consensus state: current_term: 1 leader_uuid: "64e0031dad294efda5c77af61823296d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e0031dad294efda5c77af61823296d" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35047 } } }
I20260808 23:00:02.001067 30001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:02.001147 30000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64e0031dad294efda5c77af61823296d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e0031dad294efda5c77af61823296d" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35047 } } }
I20260808 23:00:02.001287 30000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d [sys.catalog]: This master's current role is: LEADER
W20260808 23:00:02.003218 30032 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:00:02.003276 30032 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:00:02.003321 30007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:00:02.003456 30007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:00:02.004976 30007 catalog_manager.cc:1347] Generated new cluster ID: 1bb85df1fcce44969f0c112d411118dd
I20260808 23:00:02.005028 30007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 23:00:02 dist-test-slave-f0tn krb5kdc[29863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:02.011171 30016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:02.001541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57080 (local address 127.23.194.62:35047)
0808 23:00:02.001698 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:02.001707 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:02.001758 (+ 51us) server_negotiation.cc:407] Connection header received
0808 23:00:02.001853 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:02.001858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:02.001911 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:02.002022 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:02.002829 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:02.003873 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:02.004816 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:02.005336 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:02.005456 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:02.007656 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:02.007660 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:02.007661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:02.007672 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:02.009814 (+ 2142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:02.010118 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:02.010121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:02.010122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:02.010211 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:02.010386 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:02.010386 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:02.010387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:02.010604 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:02.010760 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:02.010853 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 23:00:02.010984 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":86,"threads_started":1}
I20260808 23:00:02.012131 30007 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:00:02.012773 30007 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:00:02.026742 30007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64e0031dad294efda5c77af61823296d: Generated new TSK 0
I20260808 23:00:02.026935 30007 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
I20260808 23:00:02.060001 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35047
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:43949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:02.144721 30037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:02.144936 30037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:02.144959 30037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:02.146157 30037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:02.146200 30037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:02.146226 30037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:00:02.147744 30037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:43949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:02.147985 30037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:02.148207 30037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:02.153072 30054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:02.153095 30046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:02.153203 30037 server_base.cc:1034] running on GCE node
W20260808 23:00:02.153057 30049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:02.153522 30037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:02.153759 30037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:02.154884 30037 hybrid_clock.cc:648] HybridClock initialized: now 1786230002154875 us; error 28 us; skew 500 ppm
Aug 08 23:00:02 dist-test-slave-f0tn krb5kdc[29863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230002, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:02.157307 30037 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 23:00:02.157728 30037 webserver.cc:460] Webserver started at http://127.23.194.1:44201/ using document root <none> and password file <none>
I20260808 23:00:02.157924 30037 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:02.157964 30037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:02.158054 30037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:02.159022 30037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "1ce5c80476444929829e4452f9c38f7b"
format_stamp: "Formatted at 2026-08-08 23:00:02 on dist-test-slave-f0tn"
server_key: "fcedc86faa456fac90263cb60553cffc"
server_key_iv: "f2089dc8303c206a7e753a4bcace141b"
server_key_version: "encryptionkey@0"
I20260808 23:00:02.159307 30037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "1ce5c80476444929829e4452f9c38f7b"
format_stamp: "Formatted at 2026-08-08 23:00:02 on dist-test-slave-f0tn"
server_key: "fcedc86faa456fac90263cb60553cffc"
server_key_iv: "f2089dc8303c206a7e753a4bcace141b"
server_key_version: "encryptionkey@0"
I20260808 23:00:02.160457 30037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:00:02.161092 30066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:02.161229 30037 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:02.161312 30037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "1ce5c80476444929829e4452f9c38f7b"
format_stamp: "Formatted at 2026-08-08 23:00:02 on dist-test-slave-f0tn"
server_key: "fcedc86faa456fac90263cb60553cffc"
server_key_iv: "f2089dc8303c206a7e753a4bcace141b"
server_key_version: "encryptionkey@0"
I20260808 23:00:02.161370 30037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:02.174387 30037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:02.176549 30037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:02.176690 30037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:02.176901 30037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:02.177205 30037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:02.177240 30037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:02.177270 30037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:02.177296 30037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:02.182806 30037 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:41721
I20260808 23:00:02.182844 30234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:41721 every 8 connection(s)
I20260808 23:00:02.183136 30037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:00:02.186053 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 30037
I20260808 23:00:02.186141 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:00:02.186312 24328 external_mini_cluster.cc:1428] Setting key d6c7e245806f4586ba0c169c2f79e5d6
Aug 08 23:00:02 dist-test-slave-f0tn krb5kdc[29863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230002, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:02.192699 30016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:02.183788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:38409 (local address 127.23.194.62:35047)
0808 23:00:02.183834 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:02.183837 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:02.184031 (+ 194us) server_negotiation.cc:407] Connection header received
0808 23:00:02.184943 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:02.184944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:02.184959 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:02.185010 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:02.185686 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:02.186233 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:02.187447 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:02.187664 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:02.187719 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:02.189929 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:02.189931 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:02.189931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:02.189938 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:02.191416 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:02.191787 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:02.191788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:02.191789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:02.191859 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:02.192071 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:02.192072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:02.192072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:02.192225 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:02.192296 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:02.192514 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 23:00:02.192606 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 23:00:02.192817 30239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:02.183768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35047 (local address 127.23.194.1:38409)
0808 23:00:02.183918 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:02.183953 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:02.184838 (+ 885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:02.185077 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:02.185081 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:02.185352 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:02.185587 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:02.185592 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:02.186320 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:02.186322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:02.187368 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:02.187370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:02.187732 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:02.187734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:02.187871 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:02.188313 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:02.188325 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:02.189843 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:02.191497 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:02.191503 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:02.191505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:02.191717 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:02.191919 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:02.191919 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:02.191920 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:02.192016 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:02.192282 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:02.192284 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:02.192396 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 23:00:02.192473 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 23:00:02.192588 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
I20260808 23:00:02.193287 30236 heartbeater.cc:347] Connected to a master server at 127.23.194.62:35047
I20260808 23:00:02.193405 30236 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:02.193588 30236 heartbeater.cc:511] Master 127.23.194.62:35047 requested a full tablet report, sending...
I20260808 23:00:02.194195 29936 ts_manager.cc:194] Registered new tserver with Master: 1ce5c80476444929829e4452f9c38f7b (127.23.194.1:41721)
WARNING: no policy specified for kudu/127.23.194.2@KRBTEST.COM; defaulting to no policy
I20260808 23:00:02.195135 29936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:38409
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
I20260808 23:00:02.211375 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35047
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:43949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:02.292855 30245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:02.293073 30245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:02.293095 30245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:02.294245 30245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:02.294283 30245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:02.294310 30245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:00:02.295801 30245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:43949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:02.296018 30245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:02.296222 30245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:02.301107 30252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:02.301127 30255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:02.301105 30251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:02.301368 30245 server_base.cc:1034] running on GCE node
I20260808 23:00:02.301608 30245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:02.301863 30245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:02.303056 30245 hybrid_clock.cc:648] HybridClock initialized: now 1786230002303032 us; error 36 us; skew 500 ppm
Aug 08 23:00:02 dist-test-slave-f0tn krb5kdc[29863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230002, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:02.305400 30245 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 23:00:02.305810 30245 webserver.cc:460] Webserver started at http://127.23.194.2:44963/ using document root <none> and password file <none>
I20260808 23:00:02.306017 30245 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:02.306066 30245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:02.306185 30245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:02.307216 30245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "a70d0688055349bab458c771de824733"
format_stamp: "Formatted at 2026-08-08 23:00:02 on dist-test-slave-f0tn"
server_key: "5a26e49445b937dbc61b08b0311fdfd9"
server_key_iv: "b9fc47744561d7b071a7a17e1646b599"
server_key_version: "encryptionkey@0"
I20260808 23:00:02.307528 30245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "a70d0688055349bab458c771de824733"
format_stamp: "Formatted at 2026-08-08 23:00:02 on dist-test-slave-f0tn"
server_key: "5a26e49445b937dbc61b08b0311fdfd9"
server_key_iv: "b9fc47744561d7b071a7a17e1646b599"
server_key_version: "encryptionkey@0"
I20260808 23:00:02.308713 30245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:00:02.309350 30266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:02.309533 30245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:02.309608 30245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "a70d0688055349bab458c771de824733"
format_stamp: "Formatted at 2026-08-08 23:00:02 on dist-test-slave-f0tn"
server_key: "5a26e49445b937dbc61b08b0311fdfd9"
server_key_iv: "b9fc47744561d7b071a7a17e1646b599"
server_key_version: "encryptionkey@0"
I20260808 23:00:02.309681 30245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:02.343330 30245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:02.345134 30245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:02.345266 30245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:02.345544 30245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:02.345871 30245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:02.345906 30245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:02.345937 30245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:02.345958 30245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:02.351503 30245 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:45975
I20260808 23:00:02.351547 30434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:45975 every 8 connection(s)
I20260808 23:00:02.351882 30245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:00:02.356751 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 30245
I20260808 23:00:02.356849 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:00:02.357012 24328 external_mini_cluster.cc:1428] Setting key 700ccebe6f931df1ec31229a1b35f5f3
Aug 08 23:00:02 dist-test-slave-f0tn krb5kdc[29863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230002, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:02.361238 30016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:02.352576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:58991 (local address 127.23.194.62:35047)
0808 23:00:02.352637 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:02.352639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:02.352823 (+ 184us) server_negotiation.cc:407] Connection header received
0808 23:00:02.353791 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:02.353793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:02.353807 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:02.353858 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:02.354467 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:02.354971 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:02.355937 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:02.356144 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:02.356201 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:02.358599 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:02.358600 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:02.358600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:02.358607 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:02.360018 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:02.360391 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:02.360392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:02.360392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:02.360456 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:02.360653 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:02.360654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:02.360654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:02.360782 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:02.360846 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:02.361060 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 23:00:02.361152 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 23:00:02.361308 30437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:02.352541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35047 (local address 127.23.194.2:58991)
0808 23:00:02.352710 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:02.352747 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:02.353686 (+ 939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:02.353924 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:02.353927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:02.354185 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:02.354371 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:02.354376 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:02.355058 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:02.355059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:02.355847 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:02.355849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:02.356216 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:02.356218 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:02.356336 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:02.356863 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:02.356874 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:02.358513 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:02.360099 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:02.360106 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:02.360108 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:02.360325 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:02.360513 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:02.360513 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:02.360513 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:02.360603 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:02.360836 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:02.360839 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:02.360948 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 23:00:02.361023 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 23:00:02.361141 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":89,"threads_started":1}
I20260808 23:00:02.361915 30435 heartbeater.cc:347] Connected to a master server at 127.23.194.62:35047
I20260808 23:00:02.362002 30435 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:02.362176 30435 heartbeater.cc:511] Master 127.23.194.62:35047 requested a full tablet report, sending...
I20260808 23:00:02.362562 29936 ts_manager.cc:194] Registered new tserver with Master: a70d0688055349bab458c771de824733 (127.23.194.2:45975)
I20260808 23:00:02.363042 29936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:58991
WARNING: no policy specified for kudu/127.23.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
I20260808 23:00:02.380730 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35047
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:43949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:02.460235 30441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:02.460425 30441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:02.460446 30441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:02.461580 30441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:02.461619 30441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:02.461650 30441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:00:02.463130 30441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:43949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:02.463310 30441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:02.463475 30441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:02.468194 30458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:02.468194 30463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:02.468269 30459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:02.468467 30441 server_base.cc:1034] running on GCE node
I20260808 23:00:02.468672 30441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:02.468909 30441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:02.470043 30441 hybrid_clock.cc:648] HybridClock initialized: now 1786230002470025 us; error 30 us; skew 500 ppm
Aug 08 23:00:02 dist-test-slave-f0tn krb5kdc[29863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230002, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:02.472415 30441 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 23:00:02.472828 30441 webserver.cc:460] Webserver started at http://127.23.194.3:44589/ using document root <none> and password file <none>
I20260808 23:00:02.473032 30441 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:02.473079 30441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:02.473206 30441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:02.474249 30441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "0b5db4b13a5d48b9a08d54c6e01ffbee"
format_stamp: "Formatted at 2026-08-08 23:00:02 on dist-test-slave-f0tn"
server_key: "f967ddd92674034a98a3cef1c69270f0"
server_key_iv: "a8163389662d9506e050520f24de2399"
server_key_version: "encryptionkey@0"
I20260808 23:00:02.474575 30441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "0b5db4b13a5d48b9a08d54c6e01ffbee"
format_stamp: "Formatted at 2026-08-08 23:00:02 on dist-test-slave-f0tn"
server_key: "f967ddd92674034a98a3cef1c69270f0"
server_key_iv: "a8163389662d9506e050520f24de2399"
server_key_version: "encryptionkey@0"
I20260808 23:00:02.475919 30441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:00:02.476593 30477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:02.476744 30441 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:02.476827 30441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "0b5db4b13a5d48b9a08d54c6e01ffbee"
format_stamp: "Formatted at 2026-08-08 23:00:02 on dist-test-slave-f0tn"
server_key: "f967ddd92674034a98a3cef1c69270f0"
server_key_iv: "a8163389662d9506e050520f24de2399"
server_key_version: "encryptionkey@0"
I20260808 23:00:02.476894 30441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:02.490666 30441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:02.492769 30441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:02.492901 30441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:02.493119 30441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:02.493462 30441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:02.493501 30441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:02.493532 30441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:02.493553 30441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:02.499150 30441 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:37979
I20260808 23:00:02.499187 30651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:37979 every 8 connection(s)
I20260808 23:00:02.499490 30441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 23:00:02.505363 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 30441
I20260808 23:00:02.505510 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:00:02.505738 24328 external_mini_cluster.cc:1428] Setting key d34df7f30c5e2960b289e4dbecb85ada
Aug 08 23:00:02 dist-test-slave-f0tn krb5kdc[29863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230002, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:02.509068 30016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:02.500210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:54723 (local address 127.23.194.62:35047)
0808 23:00:02.500273 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:02.500275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:02.500491 (+ 216us) server_negotiation.cc:407] Connection header received
0808 23:00:02.501558 (+ 1067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:02.501560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:02.501570 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:02.501622 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:02.502311 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:02.502868 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:02.503858 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:02.504046 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:02.504102 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:02.506517 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:02.506518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:02.506519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:02.506526 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:02.507860 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:02.508270 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:02.508271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:02.508271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:02.508332 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:02.508521 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:02.508521 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:02.508522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:02.508621 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:02.508668 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:02.508937 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 23:00:02.509001 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 23:00:02.509260 30654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:02.500197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35047 (local address 127.23.194.3:54723)
0808 23:00:02.500369 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:02.500408 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:02.501443 (+ 1035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:02.501688 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:02.501692 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:02.502006 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:02.502216 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:02.502221 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:02.502958 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:02.502959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:02.503772 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:02.503775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:02.504114 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:02.504115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:02.504232 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:02.504708 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:02.504716 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:02.506375 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:02.507932 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:02.507938 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:02.507940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:02.508192 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:02.508387 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:02.508387 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:02.508388 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:02.508479 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:02.508702 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:02.508708 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:02.508831 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 23:00:02.508921 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 23:00:02.509044 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260808 23:00:02.509711 30652 heartbeater.cc:347] Connected to a master server at 127.23.194.62:35047
I20260808 23:00:02.509807 30652 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:02.509981 30652 heartbeater.cc:511] Master 127.23.194.62:35047 requested a full tablet report, sending...
I20260808 23:00:02.510295 29936 ts_manager.cc:194] Registered new tserver with Master: 0b5db4b13a5d48b9a08d54c6e01ffbee (127.23.194.3:37979)
I20260808 23:00:02.510644 29936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:54723
I20260808 23:00:02.517469 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 23:00:02 dist-test-slave-f0tn krb5kdc[29863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 23:00:02 dist-test-slave-f0tn krb5kdc[29863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.1@KRBTEST.COM
I20260808 23:00:02.539575 30664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:02.532704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38022 (local address 127.23.194.1:41721)
0808 23:00:02.532869 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:02.532874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:02.532885 (+ 11us) server_negotiation.cc:407] Connection header received
0808 23:00:02.532912 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:02.532913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:02.532928 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:02.532994 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:02.533536 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:02.534241 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:02.534934 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:02.535259 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:02.535309 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:02.537362 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:02.537370 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:02.537370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:02.537419 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:02.538684 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:02.538966 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:02.538968 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:02.538969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:02.539019 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:02.539141 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:02.539141 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:02.539142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:02.539267 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:02.539363 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:02.539414 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 23:00:02.539474 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
Aug 08 23:00:02 dist-test-slave-f0tn krb5kdc[29863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:02.546931 30016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:02.540871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57094 (local address 127.23.194.62:35047)
0808 23:00:02.540918 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:02.540919 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:02.540941 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:00:02.540963 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:02.540963 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:02.540972 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:02.541008 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:02.541426 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:02.541903 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:02.542505 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:02.542679 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:02.542720 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:02.544645 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:02.544647 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:02.544647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:02.544653 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:02.545899 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:02.546195 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:02.546196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:02.546197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:02.546248 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:02.546408 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:02.546408 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:02.546409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:02.546551 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:02.546638 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:02.546759 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 23:00:02.546841 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 23:00:02.547142 29936 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57094
I20260808 23:00:02.547446 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 30037
I20260808 23:00:02.562916 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 30245
I20260808 23:00:02.583482 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 30441
I20260808 23:00:02.596155 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 29885
2026-08-08T23:00:02Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1980 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:00:02 dist-test-slave-f0tn krb5kdc[30671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:00:02 dist-test-slave-f0tn krb5kdc[30671](info): set up 2 sockets
Aug 08 23:00:02 dist-test-slave-f0tn krb5kdc[30671](info): commencing operation
krb5kdc: starting...
W20260808 23:00:03.732393 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:00:03 dist-test-slave-f0tn krb5kdc[30671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:00:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:00:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:00:03.849788 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:44975
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:44975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:03.943384 30689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:03.943585 30689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:03.943607 30689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:03.944759 30689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:00:03.944800 30689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:03.944813 30689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:03.944835 30689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:00:03.944857 30689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:00:03.946431 30689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:44975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:44975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:03.946672 30689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:03.946913 30689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:03.951561 30699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:03.951560 30696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:03.951663 30689 server_base.cc:1034] running on GCE node
W20260808 23:00:03.951570 30701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:03.951963 30689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:03.952248 30689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:03.953434 30689 hybrid_clock.cc:648] HybridClock initialized: now 1786230003953416 us; error 28 us; skew 500 ppm
Aug 08 23:00:03 dist-test-slave-f0tn krb5kdc[30671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230003, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:03.955787 30689 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:00:03.956077 30689 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:03.956123 30689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:03.956255 30689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:03.957250 30689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "8213081a9e084f448b208f7c2eb7a210"
format_stamp: "Formatted at 2026-08-08 23:00:03 on dist-test-slave-f0tn"
server_key: "2614aacd4faed993578cc38530d7a000"
server_key_iv: "705aa22c7c10044ef1f2a28a27f1ea83"
server_key_version: "encryptionkey@0"
I20260808 23:00:03.957607 30689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "8213081a9e084f448b208f7c2eb7a210"
format_stamp: "Formatted at 2026-08-08 23:00:03 on dist-test-slave-f0tn"
server_key: "2614aacd4faed993578cc38530d7a000"
server_key_iv: "705aa22c7c10044ef1f2a28a27f1ea83"
server_key_version: "encryptionkey@0"
I20260808 23:00:03.958793 30689 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:00:03.959403 30709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:03.959564 30689 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:03.959645 30689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "8213081a9e084f448b208f7c2eb7a210"
format_stamp: "Formatted at 2026-08-08 23:00:03 on dist-test-slave-f0tn"
server_key: "2614aacd4faed993578cc38530d7a000"
server_key_iv: "705aa22c7c10044ef1f2a28a27f1ea83"
server_key_version: "encryptionkey@0"
I20260808 23:00:03.959703 30689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:03.982911 30689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:03.984685 30689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:03.984822 30689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:03.990226 30689 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:44975
I20260808 23:00:03.990254 30789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:44975 every 8 connection(s)
I20260808 23:00:03.990528 30689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:00:03.991549 30790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:03.993883 30790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210: Bootstrap starting.
I20260808 23:00:03.994426 30790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:03.994578 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 30689
I20260808 23:00:03.994683 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:00:03.994719 30790 log.cc:824] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:03.994863 24328 external_mini_cluster.cc:1428] Setting key 0c3e80e76584f3b97da6e9af1afd8a2a
I20260808 23:00:03.995451 30790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210: No bootstrap required, opened a new log
I20260808 23:00:03.997650 30790 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8213081a9e084f448b208f7c2eb7a210" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 44975 } }
I20260808 23:00:03.997788 30790 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:03.997809 30790 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8213081a9e084f448b208f7c2eb7a210, State: Initialized, Role: FOLLOWER
I20260808 23:00:03.997915 30790 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8213081a9e084f448b208f7c2eb7a210" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 44975 } }
I20260808 23:00:03.997985 30790 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:00:03.998013 30790 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:00:03.998059 30790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:03.998554 30790 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8213081a9e084f448b208f7c2eb7a210" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 44975 } }
I20260808 23:00:03.998643 30790 leader_election.cc:302] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8213081a9e084f448b208f7c2eb7a210; no voters:
I20260808 23:00:03.998812 30790 leader_election.cc:288] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:00:03.998849 30797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:03.998998 30797 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210 [term 1 LEADER]: Becoming Leader. State: Replica: 8213081a9e084f448b208f7c2eb7a210, State: Running, Role: LEADER
I20260808 23:00:03.999078 30790 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:00:03.999135 30797 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8213081a9e084f448b208f7c2eb7a210" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 44975 } }
I20260808 23:00:03.999468 30797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8213081a9e084f448b208f7c2eb7a210. Latest consensus state: current_term: 1 leader_uuid: "8213081a9e084f448b208f7c2eb7a210" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8213081a9e084f448b208f7c2eb7a210" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 44975 } } }
I20260808 23:00:03.999544 30797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:04.000299 30801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8213081a9e084f448b208f7c2eb7a210" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8213081a9e084f448b208f7c2eb7a210" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 44975 } } }
I20260808 23:00:04.000378 30801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:04.000380 30803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:00:04.000947 30803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 23:00:03 dist-test-slave-f0tn krb5kdc[30671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:04.002590 30803 catalog_manager.cc:1347] Generated new cluster ID: 2b687155d9e048df954840c0cce30d1a
I20260808 23:00:04.002655 30803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:00:04.006415 30795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:03.995388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52448 (local address 127.23.194.62:44975)
0808 23:00:03.995584 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:03.995594 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:03.995626 (+ 32us) server_negotiation.cc:407] Connection header received
0808 23:00:03.996557 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:03.996563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:03.996611 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:03.996734 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:03.997345 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:03.998382 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:03.999363 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:03.999943 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.000060 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:04.002759 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:04.002763 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:04.002763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:04.002775 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:04.004576 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:04.004921 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:04.004924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:04.004924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:04.005016 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:04.005192 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:04.005192 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:04.005193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:04.005500 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:04.005645 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:04.006094 (+ 449us) server_negotiation.cc:299] Negotiation successful
0808 23:00:04.006231 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260808 23:00:04.009148 30803 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:00:04.009909 30803 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:00:04.013974 30803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210: Generated new TSK 0
I20260808 23:00:04.014127 30803 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
I20260808 23:00:04.040863 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:44975
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:04.120808 30828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:04.121004 30828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:04.121037 30828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:04.122259 30828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:04.122303 30828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:04.122337 30828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:00:04.123834 30828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:44975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:04.124066 30828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:04.124284 30828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:04.130018 30839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:04.130031 30837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:04.130039 30836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:04.130090 30828 server_base.cc:1034] running on GCE node
I20260808 23:00:04.130390 30828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:04.130607 30828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:04.131753 30828 hybrid_clock.cc:648] HybridClock initialized: now 1786230004131741 us; error 30 us; skew 500 ppm
Aug 08 23:00:04 dist-test-slave-f0tn krb5kdc[30671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230004, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:04.134128 30828 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 23:00:04.134450 30828 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:04.134500 30828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:04.134621 30828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:04.135599 30828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "ed3ccf6bb024458f9f80c6f36a841afa"
format_stamp: "Formatted at 2026-08-08 23:00:04 on dist-test-slave-f0tn"
server_key: "a207372aa77dadc7ebed3afe705b597d"
server_key_iv: "aaa37f365ecfe3e5a603dac465f3baa8"
server_key_version: "encryptionkey@0"
I20260808 23:00:04.135917 30828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "ed3ccf6bb024458f9f80c6f36a841afa"
format_stamp: "Formatted at 2026-08-08 23:00:04 on dist-test-slave-f0tn"
server_key: "a207372aa77dadc7ebed3afe705b597d"
server_key_iv: "aaa37f365ecfe3e5a603dac465f3baa8"
server_key_version: "encryptionkey@0"
I20260808 23:00:04.137176 30828 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:00:04.137879 30846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:04.138033 30828 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:04.138114 30828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "ed3ccf6bb024458f9f80c6f36a841afa"
format_stamp: "Formatted at 2026-08-08 23:00:04 on dist-test-slave-f0tn"
server_key: "a207372aa77dadc7ebed3afe705b597d"
server_key_iv: "aaa37f365ecfe3e5a603dac465f3baa8"
server_key_version: "encryptionkey@0"
I20260808 23:00:04.138186 30828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:04.153155 30828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:04.154915 30828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:04.155050 30828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:04.155267 30828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:04.155560 30828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:04.155594 30828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:04.155624 30828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:04.155642 30828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:04.161149 30828 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:33489
I20260808 23:00:04.161202 31007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:33489 every 8 connection(s)
I20260808 23:00:04.161470 30828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:00:04.166024 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 30828
I20260808 23:00:04.166131 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:00:04.166297 24328 external_mini_cluster.cc:1428] Setting key 882d1d008d5787edc1c710d45a717357
Aug 08 23:00:04 dist-test-slave-f0tn krb5kdc[30671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230004, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:04.170603 30795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:04.162168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:48663 (local address 127.23.194.62:44975)
0808 23:00:04.162217 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:04.162219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:04.162441 (+ 222us) server_negotiation.cc:407] Connection header received
0808 23:00:04.162910 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:04.162911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:04.162921 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:04.162960 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:04.163897 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.164400 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.165316 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.165547 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.165592 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:04.167881 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:04.167883 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:04.167883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:04.167889 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:04.169345 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:04.169737 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:04.169738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:04.169739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:04.169803 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:04.170001 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:04.170002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:04.170003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:04.170132 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:04.170211 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:04.170423 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 23:00:04.170516 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 23:00:04.170676 31012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:04.162168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:44975 (local address 127.23.194.1:48663)
0808 23:00:04.162327 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:04.162364 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:04.162823 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:04.163012 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:04.163015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:04.163312 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:04.163823 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:04.163829 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.164463 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.164464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:04.165253 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:04.165255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.165603 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.165603 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:04.165709 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:04.166175 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:04.166183 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:04.167797 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:04.169447 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:04.169455 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:04.169457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:04.169674 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:04.169860 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:04.169860 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:04.169861 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:04.169949 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:04.170202 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:04.170206 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:04.170312 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 23:00:04.170389 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 23:00:04.170506 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260808 23:00:04.171121 31009 heartbeater.cc:347] Connected to a master server at 127.23.194.62:44975
I20260808 23:00:04.171214 31009 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:04.171360 31009 heartbeater.cc:511] Master 127.23.194.62:44975 requested a full tablet report, sending...
I20260808 23:00:04.171947 30737 ts_manager.cc:194] Registered new tserver with Master: ed3ccf6bb024458f9f80c6f36a841afa (127.23.194.1:33489)
I20260808 23:00:04.172855 30737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:48663
WARNING: no policy specified for kudu/127.23.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
I20260808 23:00:04.191952 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:44975
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:04.273914 31017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:04.274147 31017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:04.274173 31017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:04.275324 31017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:04.275365 31017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:04.275390 31017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:00:04.276923 31017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:44975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:04.277159 31017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:04.277357 31017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:04.282507 31024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:04.282549 31027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:04.282580 31017 server_base.cc:1034] running on GCE node
W20260808 23:00:04.282528 31025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:04.282955 31017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:04.283202 31017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:04.284343 31017 hybrid_clock.cc:648] HybridClock initialized: now 1786230004284332 us; error 30 us; skew 500 ppm
Aug 08 23:00:04 dist-test-slave-f0tn krb5kdc[30671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230004, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:04.286829 31017 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 23:00:04.287132 31017 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:04.287182 31017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:04.287307 31017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:04.288344 31017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "234c3114a77d4f50971e066c5b6f7af4"
format_stamp: "Formatted at 2026-08-08 23:00:04 on dist-test-slave-f0tn"
server_key: "f25e6b43e8ffd04e1479ba848f0c0c70"
server_key_iv: "75d3ea8cf4cbf8f6ff3ca7487dd60bc2"
server_key_version: "encryptionkey@0"
I20260808 23:00:04.288688 31017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "234c3114a77d4f50971e066c5b6f7af4"
format_stamp: "Formatted at 2026-08-08 23:00:04 on dist-test-slave-f0tn"
server_key: "f25e6b43e8ffd04e1479ba848f0c0c70"
server_key_iv: "75d3ea8cf4cbf8f6ff3ca7487dd60bc2"
server_key_version: "encryptionkey@0"
I20260808 23:00:04.289966 31017 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:00:04.290620 31034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:04.290776 31017 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:04.290863 31017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "234c3114a77d4f50971e066c5b6f7af4"
format_stamp: "Formatted at 2026-08-08 23:00:04 on dist-test-slave-f0tn"
server_key: "f25e6b43e8ffd04e1479ba848f0c0c70"
server_key_iv: "75d3ea8cf4cbf8f6ff3ca7487dd60bc2"
server_key_version: "encryptionkey@0"
I20260808 23:00:04.290922 31017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:04.299006 31017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:04.300810 31017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:04.300944 31017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:04.301149 31017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:04.301468 31017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:04.301506 31017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:04.301534 31017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:04.301558 31017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:04.307159 31017 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:41307
I20260808 23:00:04.307194 31200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:41307 every 8 connection(s)
I20260808 23:00:04.307432 31017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
Aug 08 23:00:04 dist-test-slave-f0tn krb5kdc[30671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230004, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:04.315938 30795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:04.308120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:33673 (local address 127.23.194.62:44975)
0808 23:00:04.308191 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:04.308194 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:04.308363 (+ 169us) server_negotiation.cc:407] Connection header received
0808 23:00:04.308845 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:04.308848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:04.308860 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:04.308913 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:04.309872 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.310428 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.311383 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.311641 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.311704 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:04.313656 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:04.313658 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:04.313658 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:04.313665 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:04.314872 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:04.315200 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:04.315201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:04.315201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:04.315246 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:04.315419 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:04.315420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:04.315420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:04.315512 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:04.315566 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:04.315799 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 23:00:04.315873 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 23:00:04.316015 31203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:04.308059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:44975 (local address 127.23.194.2:33673)
0808 23:00:04.308244 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:04.308285 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:04.308747 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:04.308981 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:04.308985 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:04.309250 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:04.309782 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:04.309788 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.310514 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.310515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:04.311300 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:04.311303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.311721 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.311723 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:04.311834 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:04.312232 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:04.312239 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:04.313562 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:04.314939 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:04.314945 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:04.314947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:04.315144 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:04.315289 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:04.315290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:04.315290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:04.315379 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:04.315552 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:04.315555 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:04.315664 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 23:00:04.315739 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 23:00:04.315850 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":85,"threads_started":1}
I20260808 23:00:04.316448 31201 heartbeater.cc:347] Connected to a master server at 127.23.194.62:44975
I20260808 23:00:04.316540 31201 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:04.316694 31201 heartbeater.cc:511] Master 127.23.194.62:44975 requested a full tablet report, sending...
I20260808 23:00:04.316819 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 31017
I20260808 23:00:04.316919 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:00:04.317081 30737 ts_manager.cc:194] Registered new tserver with Master: 234c3114a77d4f50971e066c5b6f7af4 (127.23.194.2:41307)
I20260808 23:00:04.317111 24328 external_mini_cluster.cc:1428] Setting key d8744169c2d5fa643e5390aea526265a
I20260808 23:00:04.317616 30737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:33673
WARNING: no policy specified for kudu/127.23.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
I20260808 23:00:04.341065 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:44975
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:04.421548 31207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:04.421780 31207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:04.421814 31207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:04.422956 31207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:04.422999 31207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:04.423034 31207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:00:04.424517 31207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:44975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:04.424769 31207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:04.424988 31207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 23:00:04.429975 31207 server_base.cc:1034] running on GCE node
W20260808 23:00:04.429942 31216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:04.429898 31213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:04.429920 31214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:04.430403 31207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:04.430629 31207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:04.431770 31207 hybrid_clock.cc:648] HybridClock initialized: now 1786230004431750 us; error 37 us; skew 500 ppm
Aug 08 23:00:04 dist-test-slave-f0tn krb5kdc[30671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230004, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:04.434199 31207 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 23:00:04.434521 31207 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:04.434573 31207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:04.434703 31207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:04.435755 31207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "66c721badb754d31aef59eab06b97eec"
format_stamp: "Formatted at 2026-08-08 23:00:04 on dist-test-slave-f0tn"
server_key: "7387dd3e3859e81925ec4c8b536ec87c"
server_key_iv: "1145b898386c620033700414ed4b011e"
server_key_version: "encryptionkey@0"
I20260808 23:00:04.436071 31207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "66c721badb754d31aef59eab06b97eec"
format_stamp: "Formatted at 2026-08-08 23:00:04 on dist-test-slave-f0tn"
server_key: "7387dd3e3859e81925ec4c8b536ec87c"
server_key_iv: "1145b898386c620033700414ed4b011e"
server_key_version: "encryptionkey@0"
I20260808 23:00:04.437249 31207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:00:04.437953 31228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:04.438105 31207 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:04.438189 31207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "66c721badb754d31aef59eab06b97eec"
format_stamp: "Formatted at 2026-08-08 23:00:04 on dist-test-slave-f0tn"
server_key: "7387dd3e3859e81925ec4c8b536ec87c"
server_key_iv: "1145b898386c620033700414ed4b011e"
server_key_version: "encryptionkey@0"
I20260808 23:00:04.438251 31207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:04.461575 31207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:04.463469 31207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:04.463600 31207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:04.463812 31207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:04.464100 31207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:04.464134 31207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:04.464167 31207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:04.464184 31207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:04.469678 31207 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:39517
I20260808 23:00:04.469728 31396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:39517 every 8 connection(s)
I20260808 23:00:04.469973 31207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
Aug 08 23:00:04 dist-test-slave-f0tn krb5kdc[30671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230004, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:04.475903 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 31207
I20260808 23:00:04.475999 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:00:04.476209 24328 external_mini_cluster.cc:1428] Setting key 59adf7141273c2330fc666a17944e256
I20260808 23:00:04.478542 30795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:04.470643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:54051 (local address 127.23.194.62:44975)
0808 23:00:04.470700 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:04.470702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:04.470897 (+ 195us) server_negotiation.cc:407] Connection header received
0808 23:00:04.471392 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:04.471393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:04.471408 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:04.471460 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:04.472367 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.472874 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.473877 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.474076 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.474131 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:04.476034 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:04.476035 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:04.476036 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:04.476045 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:04.477536 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:04.477878 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:04.477879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:04.477879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:04.477925 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:04.478087 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:04.478088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:04.478088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:04.478195 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:04.478239 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:04.478424 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 23:00:04.478479 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 23:00:04.478718 31400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:04.470619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:44975 (local address 127.23.194.3:54051)
0808 23:00:04.470789 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:04.470823 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:04.471299 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:04.471527 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:04.471530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:04.471787 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:04.472276 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:04.472281 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.472957 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.472958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:04.473802 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:04.473805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.474150 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.474151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:04.474260 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:04.474672 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:04.474679 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:04.475937 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:04.477610 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:04.477616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:04.477618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:04.477825 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:04.477966 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:04.477966 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:04.477966 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:04.478050 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:04.478232 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:04.478235 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:04.478340 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 23:00:04.478413 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 23:00:04.478527 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
I20260808 23:00:04.479141 31397 heartbeater.cc:347] Connected to a master server at 127.23.194.62:44975
I20260808 23:00:04.479239 31397 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:04.479383 31397 heartbeater.cc:511] Master 127.23.194.62:44975 requested a full tablet report, sending...
I20260808 23:00:04.479708 30737 ts_manager.cc:194] Registered new tserver with Master: 66c721badb754d31aef59eab06b97eec (127.23.194.3:39517)
I20260808 23:00:04.480062 30737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:54051
I20260808 23:00:04.488168 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:00:04.495208 30795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:04.489529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52460 (local address 127.23.194.62:44975)
0808 23:00:04.489587 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:04.489590 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:04.489734 (+ 144us) server_negotiation.cc:407] Connection header received
0808 23:00:04.489778 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:04.489779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:04.489788 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:04.489835 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:04.490395 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.490893 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.491577 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.491766 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.491813 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:04.492918 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:04.492919 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:04.492919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:04.492927 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:04.494226 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:04.494543 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:04.494544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:04.494544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:04.494596 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:04.494747 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:04.494748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:04.494748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:04.494889 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:04.494963 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:04.495038 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 23:00:04.495103 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 23:00:04.496640 30737 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52460:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:00:04.498260 30737 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:00:04.504492 31410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:04.500468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:33489 (local address 127.0.0.1:46474)
0808 23:00:04.500927 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:04.500934 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:04.500980 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:04.501232 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:04.501233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:04.501235 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:04.501296 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:04.501297 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.502535 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.502536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:04.503362 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:04.503365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.504202 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.504203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:04.504288 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:04.504303 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 23:00:04.504368 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 23:00:04.504416 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":54,"threads_started":1}
I20260808 23:00:04.504613 31413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:04.500643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53318 (local address 127.23.194.3:39517)
0808 23:00:04.500836 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:04.500841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:04.500891 (+ 50us) server_negotiation.cc:407] Connection header received
0808 23:00:04.501205 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:04.501208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:04.501227 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:04.501293 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:04.501527 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.502237 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.503808 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.504272 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.504338 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:04.504390 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:04.504502 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 23:00:04.504537 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":90,"threads_started":1}
I20260808 23:00:04.504601 31408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:04.500464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:39517 (local address 127.0.0.1:53318)
0808 23:00:04.500839 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:04.500847 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:04.501032 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:04.501398 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:04.501399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:04.501401 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:04.501445 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:04.501446 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.502331 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.502332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:04.503699 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:04.503703 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.504352 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.504353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:04.504430 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:04.504434 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:00:04.504483 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 23:00:04.504525 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":65,"thread_start_us":63,"threads_started":1}
I20260808 23:00:04.504983 31412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:04.500647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37200 (local address 127.23.194.2:41307)
0808 23:00:04.500833 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:04.500838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:04.500849 (+ 11us) server_negotiation.cc:407] Connection header received
0808 23:00:04.500889 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:04.500891 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:04.500907 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:04.500984 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:04.501148 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.502328 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.503784 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.504461 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.504517 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:04.504569 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:04.504868 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 23:00:04.504908 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":79,"threads_started":1}
I20260808 23:00:04.505070 31415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:04.500762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46474 (local address 127.23.194.1:33489)
0808 23:00:04.500928 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:04.500932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:04.500971 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:00:04.501046 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:04.501048 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:04.501065 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:04.501132 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:04.501368 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.502449 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.503444 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.504049 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.504113 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:04.504167 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:04.504955 (+ 788us) server_negotiation.cc:299] Negotiation successful
0808 23:00:04.504993 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
I20260808 23:00:04.504808 31407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:04.500464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:41307 (local address 127.0.0.1:37200)
0808 23:00:04.500627 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:04.500679 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:04.500744 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:04.501008 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:04.501011 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:04.501014 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:04.501081 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:04.501082 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.502425 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.502426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:04.503701 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:04.503703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.504606 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.504607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:04.504662 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:04.504666 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:00:04.504710 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 23:00:04.504745 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":87,"thread_start_us":62,"threads_started":1}
I20260808 23:00:04.505903 31306 tablet_service.cc:1467] Processing CreateTablet for tablet 0ae87c3cb26b4d5f8d847609182f471c (DEFAULT_TABLE table=test-table [id=c121614bf8714da09e267b369ab7f90a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:04.505903 30912 tablet_service.cc:1467] Processing CreateTablet for tablet 0ae87c3cb26b4d5f8d847609182f471c (DEFAULT_TABLE table=test-table [id=c121614bf8714da09e267b369ab7f90a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:04.505988 31111 tablet_service.cc:1467] Processing CreateTablet for tablet 0ae87c3cb26b4d5f8d847609182f471c (DEFAULT_TABLE table=test-table [id=c121614bf8714da09e267b369ab7f90a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:04.506278 30912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ae87c3cb26b4d5f8d847609182f471c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:04.506279 31306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ae87c3cb26b4d5f8d847609182f471c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:04.506311 31111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ae87c3cb26b4d5f8d847609182f471c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:04.508750 31417 tablet_bootstrap.cc:492] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4: Bootstrap starting.
I20260808 23:00:04.509395 31417 tablet_bootstrap.cc:654] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:04.509579 31423 tablet_bootstrap.cc:492] T 0ae87c3cb26b4d5f8d847609182f471c P ed3ccf6bb024458f9f80c6f36a841afa: Bootstrap starting.
I20260808 23:00:04.509594 31418 tablet_bootstrap.cc:492] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec: Bootstrap starting.
I20260808 23:00:04.510111 31423 tablet_bootstrap.cc:654] T 0ae87c3cb26b4d5f8d847609182f471c P ed3ccf6bb024458f9f80c6f36a841afa: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:04.510100 31417 log.cc:824] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:04.510251 31418 tablet_bootstrap.cc:654] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:04.510655 31423 log.cc:824] T 0ae87c3cb26b4d5f8d847609182f471c P ed3ccf6bb024458f9f80c6f36a841afa: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:04.510787 31418 log.cc:824] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:04.511441 31423 tablet_bootstrap.cc:492] T 0ae87c3cb26b4d5f8d847609182f471c P ed3ccf6bb024458f9f80c6f36a841afa: No bootstrap required, opened a new log
I20260808 23:00:04.511492 31418 tablet_bootstrap.cc:492] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec: No bootstrap required, opened a new log
I20260808 23:00:04.511508 31423 ts_tablet_manager.cc:1397] T 0ae87c3cb26b4d5f8d847609182f471c P ed3ccf6bb024458f9f80c6f36a841afa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:00:04.511562 31418 ts_tablet_manager.cc:1397] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 23:00:04.513370 31423 raft_consensus.cc:348] T 0ae87c3cb26b4d5f8d847609182f471c P ed3ccf6bb024458f9f80c6f36a841afa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234c3114a77d4f50971e066c5b6f7af4" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41307 } } peers { permanent_uuid: "ed3ccf6bb024458f9f80c6f36a841afa" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 33489 } } peers { permanent_uuid: "66c721badb754d31aef59eab06b97eec" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 39517 } }
I20260808 23:00:04.513602 31423 raft_consensus.cc:374] T 0ae87c3cb26b4d5f8d847609182f471c P ed3ccf6bb024458f9f80c6f36a841afa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:04.513667 31423 raft_consensus.cc:729] T 0ae87c3cb26b4d5f8d847609182f471c P ed3ccf6bb024458f9f80c6f36a841afa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed3ccf6bb024458f9f80c6f36a841afa, State: Initialized, Role: FOLLOWER
I20260808 23:00:04.513722 31418 raft_consensus.cc:348] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234c3114a77d4f50971e066c5b6f7af4" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41307 } } peers { permanent_uuid: "ed3ccf6bb024458f9f80c6f36a841afa" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 33489 } } peers { permanent_uuid: "66c721badb754d31aef59eab06b97eec" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 39517 } }
I20260808 23:00:04.513842 31417 tablet_bootstrap.cc:492] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4: No bootstrap required, opened a new log
I20260808 23:00:04.513839 31423 consensus_queue.cc:260] T 0ae87c3cb26b4d5f8d847609182f471c P ed3ccf6bb024458f9f80c6f36a841afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234c3114a77d4f50971e066c5b6f7af4" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41307 } } peers { permanent_uuid: "ed3ccf6bb024458f9f80c6f36a841afa" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 33489 } } peers { permanent_uuid: "66c721badb754d31aef59eab06b97eec" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 39517 } }
I20260808 23:00:04.513931 31417 ts_tablet_manager.cc:1397] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 23:00:04.514043 31418 raft_consensus.cc:374] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:04.514133 31418 raft_consensus.cc:729] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66c721badb754d31aef59eab06b97eec, State: Initialized, Role: FOLLOWER
I20260808 23:00:04.514182 31423 ts_tablet_manager.cc:1428] T 0ae87c3cb26b4d5f8d847609182f471c P ed3ccf6bb024458f9f80c6f36a841afa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 23:00:04.514367 31009 heartbeater.cc:503] Master 127.23.194.62:44975 was elected leader, sending a full tablet report...
I20260808 23:00:04.514622 31418 consensus_queue.cc:260] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234c3114a77d4f50971e066c5b6f7af4" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41307 } } peers { permanent_uuid: "ed3ccf6bb024458f9f80c6f36a841afa" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 33489 } } peers { permanent_uuid: "66c721badb754d31aef59eab06b97eec" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 39517 } }
I20260808 23:00:04.514990 31418 ts_tablet_manager.cc:1428] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 23:00:04.515136 31397 heartbeater.cc:503] Master 127.23.194.62:44975 was elected leader, sending a full tablet report...
I20260808 23:00:04.515992 31417 raft_consensus.cc:348] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234c3114a77d4f50971e066c5b6f7af4" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41307 } } peers { permanent_uuid: "ed3ccf6bb024458f9f80c6f36a841afa" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 33489 } } peers { permanent_uuid: "66c721badb754d31aef59eab06b97eec" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 39517 } }
I20260808 23:00:04.516141 31417 raft_consensus.cc:374] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:04.516173 31417 raft_consensus.cc:729] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 234c3114a77d4f50971e066c5b6f7af4, State: Initialized, Role: FOLLOWER
I20260808 23:00:04.516281 31417 consensus_queue.cc:260] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234c3114a77d4f50971e066c5b6f7af4" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41307 } } peers { permanent_uuid: "ed3ccf6bb024458f9f80c6f36a841afa" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 33489 } } peers { permanent_uuid: "66c721badb754d31aef59eab06b97eec" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 39517 } }
I20260808 23:00:04.516546 31417 ts_tablet_manager.cc:1428] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 23:00:04.516559 31201 heartbeater.cc:503] Master 127.23.194.62:44975 was elected leader, sending a full tablet report...
W20260808 23:00:04.557932 31202 tablet.cc:2369] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:00:04.644806 31431 raft_consensus.cc:482] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:00:04.644932 31431 raft_consensus.cc:504] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234c3114a77d4f50971e066c5b6f7af4" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41307 } } peers { permanent_uuid: "ed3ccf6bb024458f9f80c6f36a841afa" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 33489 } } peers { permanent_uuid: "66c721badb754d31aef59eab06b97eec" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 39517 } }
I20260808 23:00:04.645303 31431 leader_election.cc:288] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed3ccf6bb024458f9f80c6f36a841afa (127.23.194.1:33489), 66c721badb754d31aef59eab06b97eec (127.23.194.3:39517)
I20260808 23:00:04.648138 31203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:04.645479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:39517 (local address 127.23.194.2:37041)
0808 23:00:04.645527 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:04.645540 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:04.645583 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:04.645771 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:04.645772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:04.645775 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:04.645823 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:04.645824 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.646553 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.646554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:04.647353 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:04.647355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.647917 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.647918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:04.647981 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:04.647984 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:00:04.648035 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 23:00:04.648069 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 23:00:04.648177 31413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:04.645597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:37041 (local address 127.23.194.3:39517)
0808 23:00:04.645647 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:04.645650 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:04.645665 (+ 15us) server_negotiation.cc:407] Connection header received
0808 23:00:04.645690 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:04.645691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:04.645700 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:04.645755 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:04.645902 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.646462 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.647443 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.647840 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.647893 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:04.647901 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:04.648051 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 23:00:04.648089 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 23:00:04.648231 31415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:04.645561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:41691 (local address 127.23.194.1:33489)
0808 23:00:04.645606 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:04.645607 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:04.645900 (+ 293us) server_negotiation.cc:407] Connection header received
0808 23:00:04.645961 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:04.645962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:04.645970 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:04.646002 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:04.646150 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.646642 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.647551 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.647910 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.647978 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:04.647986 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:04.648130 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 23:00:04.648170 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 23:00:04.648245 31432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:04.645655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:33489 (local address 127.23.194.2:41691)
0808 23:00:04.645846 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:04.645856 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:04.645908 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:04.646016 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:04.646016 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:04.646018 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:04.646082 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:04.646084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.646716 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.646717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:04.647468 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:04.647470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.647984 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.647985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:04.648058 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:04.648062 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:00:04.648113 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 23:00:04.648150 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":117,"threads_started":1}
I20260808 23:00:04.648700 30943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ae87c3cb26b4d5f8d847609182f471c" candidate_uuid: "234c3114a77d4f50971e066c5b6f7af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed3ccf6bb024458f9f80c6f36a841afa" is_pre_election: true
I20260808 23:00:04.648738 31337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ae87c3cb26b4d5f8d847609182f471c" candidate_uuid: "234c3114a77d4f50971e066c5b6f7af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66c721badb754d31aef59eab06b97eec" is_pre_election: true
I20260808 23:00:04.648838 30943 raft_consensus.cc:2454] T 0ae87c3cb26b4d5f8d847609182f471c P ed3ccf6bb024458f9f80c6f36a841afa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 234c3114a77d4f50971e066c5b6f7af4 in term 0.
I20260808 23:00:04.648869 31337 raft_consensus.cc:2454] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 234c3114a77d4f50971e066c5b6f7af4 in term 0.
I20260808 23:00:04.649078 31042 leader_election.cc:302] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 234c3114a77d4f50971e066c5b6f7af4, ed3ccf6bb024458f9f80c6f36a841afa; no voters:
I20260808 23:00:04.649318 31431 raft_consensus.cc:2781] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:00:04.649372 31431 raft_consensus.cc:482] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:00:04.649410 31431 raft_consensus.cc:3037] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:04.650044 31431 raft_consensus.cc:504] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234c3114a77d4f50971e066c5b6f7af4" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41307 } } peers { permanent_uuid: "ed3ccf6bb024458f9f80c6f36a841afa" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 33489 } } peers { permanent_uuid: "66c721badb754d31aef59eab06b97eec" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 39517 } }
I20260808 23:00:04.650158 31431 leader_election.cc:288] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [CANDIDATE]: Term 1 election: Requested vote from peers ed3ccf6bb024458f9f80c6f36a841afa (127.23.194.1:33489), 66c721badb754d31aef59eab06b97eec (127.23.194.3:39517)
I20260808 23:00:04.650301 31337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ae87c3cb26b4d5f8d847609182f471c" candidate_uuid: "234c3114a77d4f50971e066c5b6f7af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66c721badb754d31aef59eab06b97eec"
I20260808 23:00:04.650329 30943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ae87c3cb26b4d5f8d847609182f471c" candidate_uuid: "234c3114a77d4f50971e066c5b6f7af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed3ccf6bb024458f9f80c6f36a841afa"
I20260808 23:00:04.650377 31337 raft_consensus.cc:3037] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:04.650385 30943 raft_consensus.cc:3037] T 0ae87c3cb26b4d5f8d847609182f471c P ed3ccf6bb024458f9f80c6f36a841afa [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:04.651054 30943 raft_consensus.cc:2454] T 0ae87c3cb26b4d5f8d847609182f471c P ed3ccf6bb024458f9f80c6f36a841afa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 234c3114a77d4f50971e066c5b6f7af4 in term 1.
I20260808 23:00:04.651105 31337 raft_consensus.cc:2454] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 234c3114a77d4f50971e066c5b6f7af4 in term 1.
I20260808 23:00:04.651209 31042 leader_election.cc:302] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 234c3114a77d4f50971e066c5b6f7af4, ed3ccf6bb024458f9f80c6f36a841afa; no voters:
I20260808 23:00:04.651295 31431 raft_consensus.cc:2781] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:04.651458 31431 raft_consensus.cc:686] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [term 1 LEADER]: Becoming Leader. State: Replica: 234c3114a77d4f50971e066c5b6f7af4, State: Running, Role: LEADER
I20260808 23:00:04.651551 31431 consensus_queue.cc:237] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234c3114a77d4f50971e066c5b6f7af4" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41307 } } peers { permanent_uuid: "ed3ccf6bb024458f9f80c6f36a841afa" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 33489 } } peers { permanent_uuid: "66c721badb754d31aef59eab06b97eec" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 39517 } }
I20260808 23:00:04.652376 30737 catalog_manager.cc:5495] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 234c3114a77d4f50971e066c5b6f7af4 (127.23.194.2). New cstate: current_term: 1 leader_uuid: "234c3114a77d4f50971e066c5b6f7af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234c3114a77d4f50971e066c5b6f7af4" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed3ccf6bb024458f9f80c6f36a841afa" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 33489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66c721badb754d31aef59eab06b97eec" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 39517 } health_report { overall_health: UNKNOWN } } }
W20260808 23:00:04.662123 31010 tablet.cc:2369] T 0ae87c3cb26b4d5f8d847609182f471c P ed3ccf6bb024458f9f80c6f36a841afa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:00:04.684281 31412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:04.682247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47138 (local address 127.23.194.2:41307)
0808 23:00:04.682304 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:04.682305 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:04.682313 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:00:04.682348 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:04.682349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:04.682367 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:04.682427 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:00:04.682539 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.682995 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.683674 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:04.683852 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:04.683898 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:04.684002 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:00:04.684104 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:00:04.684166 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:04.684177 (+ 11us) server_negotiation.cc:299] Negotiation successful
0808 23:00:04.684193 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 23:00:04.686281 31337 raft_consensus.cc:1261] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec [term 1 FOLLOWER]: Refusing update from remote peer 234c3114a77d4f50971e066c5b6f7af4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:00:04.686280 30943 raft_consensus.cc:1261] T 0ae87c3cb26b4d5f8d847609182f471c P ed3ccf6bb024458f9f80c6f36a841afa [term 1 FOLLOWER]: Refusing update from remote peer 234c3114a77d4f50971e066c5b6f7af4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:00:04.686691 31433 consensus_queue.cc:1035] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed3ccf6bb024458f9f80c6f36a841afa" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 33489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:04.686820 31431 consensus_queue.cc:1035] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66c721badb754d31aef59eab06b97eec" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 39517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:04.688952 31436 mvcc.cc:205] Tried to move back new op lower bound from 7316398099192455168 to 7316398099053268992. Current Snapshot: MvccSnapshot[applied={T|T < 7316398099192455168}]
I20260808 23:00:04.689132 31437 mvcc.cc:205] Tried to move back new op lower bound from 7316398099192455168 to 7316398099053268992. Current Snapshot: MvccSnapshot[applied={T|T < 7316398099192455168}]
I20260808 23:00:04.689424 31439 mvcc.cc:205] Tried to move back new op lower bound from 7316398099192455168 to 7316398099053268992. Current Snapshot: MvccSnapshot[applied={T|T < 7316398099192455168}]
I20260808 23:00:04.691280 30737 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:00:04.691363 30737 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:00:04.692061 30737 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8213081a9e084f448b208f7c2eb7a210: Sending DeleteTablet for 3 replicas of tablet 0ae87c3cb26b4d5f8d847609182f471c
I20260808 23:00:04.692353 31111 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ae87c3cb26b4d5f8d847609182f471c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:04 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:37200
I20260808 23:00:04.692471 30912 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ae87c3cb26b4d5f8d847609182f471c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:04 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:46474
I20260808 23:00:04.692551 31454 tablet_replica.cc:330] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4: stopping tablet replica
I20260808 23:00:04.692646 31454 raft_consensus.cc:2229] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:00:04.692678 31306 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ae87c3cb26b4d5f8d847609182f471c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:04 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:53318
I20260808 23:00:04.692736 31454 raft_consensus.cc:2258] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:04.692762 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 30828
I20260808 23:00:04.700493 31458 tablet_replica.cc:330] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec: stopping tablet replica
I20260808 23:00:04.700618 31458 raft_consensus.cc:2229] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:00:04.700763 31458 raft_consensus.cc:2258] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:04.703624 31458 ts_tablet_manager.cc:1905] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:04.703912 31454 ts_tablet_manager.cc:1905] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:04.705286 31458 ts_tablet_manager.cc:1918] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:00:04.705360 31458 log.cc:1196] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/0ae87c3cb26b4d5f8d847609182f471c
I20260808 23:00:04.705502 31454 ts_tablet_manager.cc:1918] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:00:04.705570 31454 log.cc:1196] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/0ae87c3cb26b4d5f8d847609182f471c
I20260808 23:00:04.705664 31458 ts_tablet_manager.cc:1939] T 0ae87c3cb26b4d5f8d847609182f471c P 66c721badb754d31aef59eab06b97eec: Deleting consensus metadata
I20260808 23:00:04.705826 31454 ts_tablet_manager.cc:1939] T 0ae87c3cb26b4d5f8d847609182f471c P 234c3114a77d4f50971e066c5b6f7af4: Deleting consensus metadata
I20260808 23:00:04.706099 30710 catalog_manager.cc:4841] TS 66c721badb754d31aef59eab06b97eec (127.23.194.3:39517): tablet 0ae87c3cb26b4d5f8d847609182f471c (table test-table [id=c121614bf8714da09e267b369ab7f90a]) successfully deleted
I20260808 23:00:04.706219 30716 catalog_manager.cc:4841] TS 234c3114a77d4f50971e066c5b6f7af4 (127.23.194.2:41307): tablet 0ae87c3cb26b4d5f8d847609182f471c (table test-table [id=c121614bf8714da09e267b369ab7f90a]) successfully deleted
W20260808 23:00:04.708321 30717 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.194.1:33489 (error 108)
I20260808 23:00:04.708520 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 31017
I20260808 23:00:04.730546 31408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:04.730355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:33489 (local address 127.0.0.1:56774)
0808 23:00:04.730414 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:04.730493 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.1:33489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 23:00:04.730760 30717 catalog_manager.cc:4584] TS ed3ccf6bb024458f9f80c6f36a841afa (127.23.194.1:33489): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0ae87c3cb26b4d5f8d847609182f471c: Network error: Client connection negotiation failed: client connection to 127.23.194.1:33489: connect: Connection refused (error 111)
I20260808 23:00:04.734369 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 31207
I20260808 23:00:04.747225 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 30689
2026-08-08T23:00:04Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2146 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T23:00:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:00:04Z Disabled control of system clock
I20260808 23:00:04.779560 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:42601
--webserver_interface=127.23.194.62
--webserver_port=0
--builtin_ntp_servers=127.23.194.20:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:42601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 23:00:04.861227 31466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:04.861464 31466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:04.861486 31466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:04.862640 31466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:00:04.862679 31466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:04.862690 31466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:04.862701 31466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:00:04.862718 31466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:00:04.864045 31466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:42601
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:42601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:04.864269 31466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:04.864522 31466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:04.869407 31476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:04.869395 31478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:04.869626 31482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:04.869506 31466 server_base.cc:1034] running on GCE node
I20260808 23:00:04.869884 31466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:04.870175 31466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:04.871323 31466 hybrid_clock.cc:648] HybridClock initialized: now 1786230004871304 us; error 30 us; skew 500 ppm
I20260808 23:00:04.872556 31466 webserver.cc:460] Webserver started at http://127.23.194.62:36561/ using document root <none> and password file <none>
I20260808 23:00:04.872795 31466 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:04.872841 31466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:04.872939 31466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:04.874003 31466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "dbf1299f7813423b923c1c358286e5c8"
format_stamp: "Formatted at 2026-08-08 23:00:04 on dist-test-slave-f0tn"
server_key: "458999f27171aada0cdad8eecc1ef5d2"
server_key_iv: "756063c12ea173da26936606f200f560"
server_key_version: "encryptionkey@0"
I20260808 23:00:04.874300 31466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "dbf1299f7813423b923c1c358286e5c8"
format_stamp: "Formatted at 2026-08-08 23:00:04 on dist-test-slave-f0tn"
server_key: "458999f27171aada0cdad8eecc1ef5d2"
server_key_iv: "756063c12ea173da26936606f200f560"
server_key_version: "encryptionkey@0"
I20260808 23:00:04.875573 31466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:00:04.876217 31492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:04.876365 31466 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:04.876451 31466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "dbf1299f7813423b923c1c358286e5c8"
format_stamp: "Formatted at 2026-08-08 23:00:04 on dist-test-slave-f0tn"
server_key: "458999f27171aada0cdad8eecc1ef5d2"
server_key_iv: "756063c12ea173da26936606f200f560"
server_key_version: "encryptionkey@0"
I20260808 23:00:04.876511 31466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:04.879830 31466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:04.880088 31466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:04.880213 31466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:04.884302 31466 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:42601
I20260808 23:00:04.884438 31574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:42601 every 8 connection(s)
I20260808 23:00:04.884678 31466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:00:04.885957 31575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:04.888782 31575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8: Bootstrap starting.
I20260808 23:00:04.889351 31575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:04.889652 31575 log.cc:824] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:04.890296 31575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8: No bootstrap required, opened a new log
I20260808 23:00:04.891819 31575 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1299f7813423b923c1c358286e5c8" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42601 } }
I20260808 23:00:04.891952 31575 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:04.891985 31575 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbf1299f7813423b923c1c358286e5c8, State: Initialized, Role: FOLLOWER
I20260808 23:00:04.892127 31575 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1299f7813423b923c1c358286e5c8" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42601 } }
I20260808 23:00:04.892208 31575 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:00:04.892242 31575 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:00:04.892288 31575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:04.892789 31575 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1299f7813423b923c1c358286e5c8" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42601 } }
I20260808 23:00:04.892879 31575 leader_election.cc:302] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbf1299f7813423b923c1c358286e5c8; no voters:
I20260808 23:00:04.893029 31575 leader_election.cc:288] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:00:04.893072 31579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:04.893255 31579 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8 [term 1 LEADER]: Becoming Leader. State: Replica: dbf1299f7813423b923c1c358286e5c8, State: Running, Role: LEADER
I20260808 23:00:04.893258 31575 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:00:04.893407 31579 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1299f7813423b923c1c358286e5c8" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42601 } }
I20260808 23:00:04.893572 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 31466
I20260808 23:00:04.893676 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:00:04.893870 24328 external_mini_cluster.cc:1428] Setting key 6fa3b3d85b5b80f026f0f2c4e634dff8
I20260808 23:00:04.894122 31581 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbf1299f7813423b923c1c358286e5c8. Latest consensus state: current_term: 1 leader_uuid: "dbf1299f7813423b923c1c358286e5c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1299f7813423b923c1c358286e5c8" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42601 } } }
I20260808 23:00:04.894200 31581 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:04.894621 31580 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbf1299f7813423b923c1c358286e5c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1299f7813423b923c1c358286e5c8" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42601 } } }
I20260808 23:00:04.894771 31580 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8 [sys.catalog]: This master's current role is: LEADER
W20260808 23:00:04.895255 31609 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:00:04.895301 31609 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:00:04.895386 31592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:00:04.895562 31592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:00:04.896965 31592 catalog_manager.cc:1347] Generated new cluster ID: 9334dea9b3f3498f85edfa905a3a4b0f
I20260808 23:00:04.897017 31596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:04.894341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47772 (local address 127.23.194.62:42601)
0808 23:00:04.894521 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:04.894531 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:04.895044 (+ 513us) server_negotiation.cc:407] Connection header received
0808 23:00:04.895302 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:04.895309 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:04.895354 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:04.895499 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:04.896549 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:04.896553 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:04.896553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:04.896560 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:04.896608 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:04.896623 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:04.896697 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:04.896765 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 23:00:04.896824 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":75,"threads_started":1}
I20260808 23:00:04.897019 31592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:00:04.907660 31592 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:00:04.907823 31592 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:00:04.911465 31592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8: Generated new TSK 0
I20260808 23:00:04.911630 31592 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:00:04.917748 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:42601
--builtin_ntp_servers=127.23.194.20:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 23:00:04.999096 31615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:04.999300 31615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:04.999330 31615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:05.000461 31615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:05.000500 31615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:05.000535 31615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:00:05.001935 31615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:42601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:05.002175 31615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:05.002465 31615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:05.008529 31622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:05.008553 31623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:05.008877 31625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:05.010241 31615 server_base.cc:1034] running on GCE node
I20260808 23:00:05.010484 31615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:05.010766 31615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:05.011950 31615 hybrid_clock.cc:648] HybridClock initialized: now 1786230005011933 us; error 34 us; skew 500 ppm
I20260808 23:00:05.013239 31615 webserver.cc:460] Webserver started at http://127.23.194.1:44065/ using document root <none> and password file <none>
I20260808 23:00:05.013525 31615 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:05.013577 31615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:05.013792 31615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:05.014858 31615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "21d9efbea7e94ce3b178138a3130a908"
format_stamp: "Formatted at 2026-08-08 23:00:05 on dist-test-slave-f0tn"
server_key: "2f1f7b3cd3f051b4ebd6e0d9aa08e12a"
server_key_iv: "1bdd9aed44f5676c03de4e400ce344da"
server_key_version: "encryptionkey@0"
I20260808 23:00:05.015192 31615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "21d9efbea7e94ce3b178138a3130a908"
format_stamp: "Formatted at 2026-08-08 23:00:05 on dist-test-slave-f0tn"
server_key: "2f1f7b3cd3f051b4ebd6e0d9aa08e12a"
server_key_iv: "1bdd9aed44f5676c03de4e400ce344da"
server_key_version: "encryptionkey@0"
I20260808 23:00:05.016438 31615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:00:05.017144 31637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:05.017314 31615 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:05.017437 31615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "21d9efbea7e94ce3b178138a3130a908"
format_stamp: "Formatted at 2026-08-08 23:00:05 on dist-test-slave-f0tn"
server_key: "2f1f7b3cd3f051b4ebd6e0d9aa08e12a"
server_key_iv: "1bdd9aed44f5676c03de4e400ce344da"
server_key_version: "encryptionkey@0"
I20260808 23:00:05.017518 31615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:05.020645 31615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:05.020857 31615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:05.020946 31615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:05.021095 31615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:05.021337 31615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:05.021364 31615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:05.021410 31615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:05.021435 31615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:05.026818 31615 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:36305
I20260808 23:00:05.026881 31811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:36305 every 8 connection(s)
I20260808 23:00:05.027153 31615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:00:05.029932 31596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.027772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:47565 (local address 127.23.194.62:42601)
0808 23:00:05.027812 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:05.027814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:05.028023 (+ 209us) server_negotiation.cc:407] Connection header received
0808 23:00:05.028496 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:05.028498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:05.028505 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:05.028558 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:05.029696 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:05.029697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:05.029698 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:05.029702 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:05.029719 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:05.029723 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:05.029773 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:05.029849 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 23:00:05.029864 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 23:00:05.030099 31814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.027752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:42601 (local address 127.23.194.1:47565)
0808 23:00:05.027914 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:05.027946 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:05.028401 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:05.028621 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:05.028625 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:05.028636 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:05.029544 (+ 908us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:00:05.029544 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:05.029572 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:00:05.029600 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:05.029781 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:05.029786 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:05.029789 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:00:05.029855 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 23:00:05.029916 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":76,"threads_started":1}
I20260808 23:00:05.030519 31812 heartbeater.cc:347] Connected to a master server at 127.23.194.62:42601
I20260808 23:00:05.030596 31812 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:05.030714 31812 heartbeater.cc:511] Master 127.23.194.62:42601 requested a full tablet report, sending...
I20260808 23:00:05.031149 31521 ts_manager.cc:194] Registered new tserver with Master: 21d9efbea7e94ce3b178138a3130a908 (127.23.194.1:36305)
I20260808 23:00:05.032930 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 31615
I20260808 23:00:05.033006 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:00:05.033150 24328 external_mini_cluster.cc:1428] Setting key 05355116f9da7b9ec1fccaf38022cb00
I20260808 23:00:05.033957 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:42601
--builtin_ntp_servers=127.23.194.20:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 23:00:05.114118 31818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:05.114322 31818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:05.114342 31818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:05.115450 31818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:05.115486 31818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:05.115510 31818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:00:05.116802 31818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:42601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:05.116987 31818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:05.117242 31818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:05.121978 31829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:05.121987 31834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:05.122006 31831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:05.122273 31818 server_base.cc:1034] running on GCE node
I20260808 23:00:05.122454 31818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:05.122695 31818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:05.123832 31818 hybrid_clock.cc:648] HybridClock initialized: now 1786230005123819 us; error 28 us; skew 500 ppm
I20260808 23:00:05.124948 31818 webserver.cc:460] Webserver started at http://127.23.194.2:46309/ using document root <none> and password file <none>
I20260808 23:00:05.125173 31818 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:05.125222 31818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:05.125349 31818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:05.126376 31818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "f6d90f1da9d041b3b9aa1e8d7da0657f"
format_stamp: "Formatted at 2026-08-08 23:00:05 on dist-test-slave-f0tn"
server_key: "c3c5bee9fa56627df317565c5e170a6c"
server_key_iv: "e6dccff34ea3cec28f48edc81ff4d444"
server_key_version: "encryptionkey@0"
I20260808 23:00:05.126701 31818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "f6d90f1da9d041b3b9aa1e8d7da0657f"
format_stamp: "Formatted at 2026-08-08 23:00:05 on dist-test-slave-f0tn"
server_key: "c3c5bee9fa56627df317565c5e170a6c"
server_key_iv: "e6dccff34ea3cec28f48edc81ff4d444"
server_key_version: "encryptionkey@0"
I20260808 23:00:05.127861 31818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:00:05.128500 31843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:05.128654 31818 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:05.128746 31818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "f6d90f1da9d041b3b9aa1e8d7da0657f"
format_stamp: "Formatted at 2026-08-08 23:00:05 on dist-test-slave-f0tn"
server_key: "c3c5bee9fa56627df317565c5e170a6c"
server_key_iv: "e6dccff34ea3cec28f48edc81ff4d444"
server_key_version: "encryptionkey@0"
I20260808 23:00:05.128808 31818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:05.132136 31818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:05.132364 31818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:05.132477 31818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:05.132666 31818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:05.132941 31818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:05.132972 31818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:05.132998 31818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:05.133021 31818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:05.138499 31818 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:40661
I20260808 23:00:05.138530 32019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:40661 every 8 connection(s)
I20260808 23:00:05.138823 31818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:00:05.141543 31596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.139445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:40431 (local address 127.23.194.62:42601)
0808 23:00:05.139489 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:05.139491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:05.139707 (+ 216us) server_negotiation.cc:407] Connection header received
0808 23:00:05.140160 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:05.140162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:05.140169 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:05.140217 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:05.141287 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:05.141288 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:05.141289 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:05.141293 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:05.141319 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:05.141324 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:05.141372 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:05.141457 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 23:00:05.141472 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 23:00:05.141739 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.139427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:42601 (local address 127.23.194.2:40431)
0808 23:00:05.139592 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:05.139624 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:05.140063 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:05.140294 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:05.140298 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:05.140308 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:05.141128 (+ 820us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:00:05.141128 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:05.141173 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:00:05.141199 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:05.141389 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:05.141395 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:05.141398 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:00:05.141468 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 23:00:05.141527 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260808 23:00:05.142164 32021 heartbeater.cc:347] Connected to a master server at 127.23.194.62:42601
I20260808 23:00:05.142251 32021 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:05.142395 32021 heartbeater.cc:511] Master 127.23.194.62:42601 requested a full tablet report, sending...
I20260808 23:00:05.142719 31521 ts_manager.cc:194] Registered new tserver with Master: f6d90f1da9d041b3b9aa1e8d7da0657f (127.23.194.2:40661)
I20260808 23:00:05.148700 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 31818
I20260808 23:00:05.148792 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:00:05.148957 24328 external_mini_cluster.cc:1428] Setting key e9ef94c3d07c4857d93d7c76743d2046
I20260808 23:00:05.149788 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:42601
--builtin_ntp_servers=127.23.194.20:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 23:00:05.234444 32025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:05.234674 32025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:05.234709 32025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:05.235901 32025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:05.235949 32025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:05.235994 32025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:00:05.237496 32025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:42601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:05.237757 32025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:05.238075 32025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:05.243063 32036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:05.243049 32032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:05.243144 32025 server_base.cc:1034] running on GCE node
W20260808 23:00:05.243038 32031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:05.243485 32025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:05.243722 32025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:05.244871 32025 hybrid_clock.cc:648] HybridClock initialized: now 1786230005244855 us; error 28 us; skew 500 ppm
I20260808 23:00:05.246093 32025 webserver.cc:460] Webserver started at http://127.23.194.3:36023/ using document root <none> and password file <none>
I20260808 23:00:05.246316 32025 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:05.246366 32025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:05.246496 32025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:05.247522 32025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "be361b70043347d2908f8fc817b7e841"
format_stamp: "Formatted at 2026-08-08 23:00:05 on dist-test-slave-f0tn"
server_key: "e412c54ef1a58fa3cdf62521bd914a10"
server_key_iv: "c1e3645c89b50831ebb509c329e8f0d4"
server_key_version: "encryptionkey@0"
I20260808 23:00:05.247859 32025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "be361b70043347d2908f8fc817b7e841"
format_stamp: "Formatted at 2026-08-08 23:00:05 on dist-test-slave-f0tn"
server_key: "e412c54ef1a58fa3cdf62521bd914a10"
server_key_iv: "c1e3645c89b50831ebb509c329e8f0d4"
server_key_version: "encryptionkey@0"
I20260808 23:00:05.249070 32025 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 23:00:05.249794 32044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:05.249938 32025 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:05.250021 32025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "be361b70043347d2908f8fc817b7e841"
format_stamp: "Formatted at 2026-08-08 23:00:05 on dist-test-slave-f0tn"
server_key: "e412c54ef1a58fa3cdf62521bd914a10"
server_key_iv: "c1e3645c89b50831ebb509c329e8f0d4"
server_key_version: "encryptionkey@0"
I20260808 23:00:05.250079 32025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:05.253227 32025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:05.253504 32025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:05.253625 32025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:05.253819 32025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:05.254096 32025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:05.254128 32025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:05.254153 32025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:05.254171 32025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:05.259599 32025 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:33157
I20260808 23:00:05.259641 32229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:33157 every 8 connection(s)
I20260808 23:00:05.259948 32025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 23:00:05.262876 31596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.260647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:34215 (local address 127.23.194.62:42601)
0808 23:00:05.260711 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:05.260714 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:05.260917 (+ 203us) server_negotiation.cc:407] Connection header received
0808 23:00:05.261424 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:05.261425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:05.261433 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:05.261486 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:05.262626 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:05.262628 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:05.262628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:05.262633 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:05.262651 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:05.262663 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:05.262711 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:05.262789 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 23:00:05.262805 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 23:00:05.263049 32232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.260604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:42601 (local address 127.23.194.3:34215)
0808 23:00:05.260805 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:05.260837 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:05.261296 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:05.261552 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:05.261556 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:05.261566 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:05.262477 (+ 911us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:00:05.262477 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:05.262504 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:00:05.262532 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:05.262716 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:05.262721 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:05.262725 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:00:05.262794 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 23:00:05.262856 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":93,"threads_started":1}
I20260808 23:00:05.263506 32230 heartbeater.cc:347] Connected to a master server at 127.23.194.62:42601
I20260808 23:00:05.263597 32230 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:05.263741 32230 heartbeater.cc:511] Master 127.23.194.62:42601 requested a full tablet report, sending...
I20260808 23:00:05.264096 31521 ts_manager.cc:194] Registered new tserver with Master: be361b70043347d2908f8fc817b7e841 (127.23.194.3:33157)
I20260808 23:00:05.264813 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 32025
I20260808 23:00:05.264899 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:00:05.265039 24328 external_mini_cluster.cc:1428] Setting key ce38ef64db8fa589e7dc0f0b97bb603a
I20260808 23:00:05.265838 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:00:05.268065 31596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.267096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47788 (local address 127.23.194.62:42601)
0808 23:00:05.267142 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:05.267144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:05.267230 (+ 86us) server_negotiation.cc:407] Connection header received
0808 23:00:05.267293 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:05.267294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:05.267298 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:05.267345 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:05.267857 (+ 512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:05.267859 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:05.267859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:05.267863 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:05.267879 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:05.267882 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:05.267928 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:05.267986 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 23:00:05.268001 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 23:00:05.269619 31521 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:00:05.271093 31521 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:00:05.274802 32247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.273210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49228 (local address 127.23.194.3:33157)
0808 23:00:05.273349 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:05.273352 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:05.273507 (+ 155us) server_negotiation.cc:407] Connection header received
0808 23:00:05.273549 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:05.273551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:05.273561 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:05.273625 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:05.274458 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:05.274466 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:05.274466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:05.274472 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:05.274507 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:05.274532 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:05.274594 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:05.274708 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 23:00:05.274725 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260808 23:00:05.274881 32243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.272997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:36305 (local address 127.0.0.1:48546)
0808 23:00:05.273303 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:05.273345 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:05.273425 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:05.273923 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:05.273924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:05.273927 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:05.274380 (+ 453us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:00:05.274381 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:05.274389 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:00:05.274401 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:05.274749 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:05.274750 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:05.274750 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 23:00:05.274801 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 23:00:05.274814 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":58,"threads_started":1}
I20260808 23:00:05.274905 32248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.273246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48546 (local address 127.23.194.1:36305)
0808 23:00:05.273454 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:05.273457 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:05.273468 (+ 11us) server_negotiation.cc:407] Connection header received
0808 23:00:05.273494 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:05.273496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:05.273506 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:05.273574 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:05.274632 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:05.274639 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:05.274639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:05.274644 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:05.274675 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:05.274693 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:05.274742 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:05.274810 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 23:00:05.274824 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":76,"threads_started":1}
I20260808 23:00:05.274950 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.273109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47134 (local address 127.23.194.2:40661)
0808 23:00:05.273271 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:05.273275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:05.273571 (+ 296us) server_negotiation.cc:407] Connection header received
0808 23:00:05.273604 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:05.273606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:05.273616 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:05.273720 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:05.274618 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:05.274627 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:05.274627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:05.274631 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:05.274679 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:05.274708 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:05.274760 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:05.274855 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 23:00:05.274872 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
I20260808 23:00:05.275012 32242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.272999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:33157 (local address 127.0.0.1:49228)
0808 23:00:05.273436 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:05.273445 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:05.273498 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:05.273912 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:05.273914 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:05.273916 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:05.274323 (+ 407us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:00:05.274324 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:05.274337 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:00:05.274354 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:05.274594 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:05.274601 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:05.274605 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:00:05.274927 (+ 322us) client_negotiation.cc:240] Negotiation successful
0808 23:00:05.274939 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":92,"threads_started":1}
I20260808 23:00:05.275606 32126 tablet_service.cc:1467] Processing CreateTablet for tablet 68c65243884e49a689e05750e410cf0b (DEFAULT_TABLE table=test-table [id=489a76e595bf49549afb6a595ebdc903]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:05.275604 31714 tablet_service.cc:1467] Processing CreateTablet for tablet 68c65243884e49a689e05750e410cf0b (DEFAULT_TABLE table=test-table [id=489a76e595bf49549afb6a595ebdc903]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:05.275940 32126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68c65243884e49a689e05750e410cf0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:05.275935 31714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68c65243884e49a689e05750e410cf0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:05.278385 32252 tablet_bootstrap.cc:492] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841: Bootstrap starting.
I20260808 23:00:05.278645 32253 tablet_bootstrap.cc:492] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908: Bootstrap starting.
I20260808 23:00:05.278694 32244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.273023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:40661 (local address 127.0.0.1:47134)
0808 23:00:05.273507 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:05.273516 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:05.273560 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:05.273725 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:05.273731 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:05.273737 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:05.274546 (+ 809us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:00:05.274546 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:05.274551 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:00:05.274559 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:05.274784 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:05.274785 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:05.274785 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 23:00:05.275274 (+ 489us) client_negotiation.cc:240] Negotiation successful
0808 23:00:05.278616 (+ 3342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":92,"threads_started":1}
I20260808 23:00:05.279110 32252 tablet_bootstrap.cc:654] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:05.279301 32253 tablet_bootstrap.cc:654] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:05.279421 32252 log.cc:824] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:05.279444 31915 tablet_service.cc:1467] Processing CreateTablet for tablet 68c65243884e49a689e05750e410cf0b (DEFAULT_TABLE table=test-table [id=489a76e595bf49549afb6a595ebdc903]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:05.279682 32253 log.cc:824] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:05.279819 31915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68c65243884e49a689e05750e410cf0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:05.280442 32253 tablet_bootstrap.cc:492] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908: No bootstrap required, opened a new log
I20260808 23:00:05.280512 32253 ts_tablet_manager.cc:1397] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:00:05.281888 32252 tablet_bootstrap.cc:492] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841: No bootstrap required, opened a new log
I20260808 23:00:05.281953 32252 ts_tablet_manager.cc:1397] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 23:00:05.283111 32257 tablet_bootstrap.cc:492] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f: Bootstrap starting.
I20260808 23:00:05.283265 32253 raft_consensus.cc:348] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be361b70043347d2908f8fc817b7e841" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33157 } } peers { permanent_uuid: "21d9efbea7e94ce3b178138a3130a908" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36305 } } peers { permanent_uuid: "f6d90f1da9d041b3b9aa1e8d7da0657f" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40661 } }
I20260808 23:00:05.283407 32253 raft_consensus.cc:374] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:05.283432 32253 raft_consensus.cc:729] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21d9efbea7e94ce3b178138a3130a908, State: Initialized, Role: FOLLOWER
I20260808 23:00:05.283565 32253 consensus_queue.cc:260] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be361b70043347d2908f8fc817b7e841" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33157 } } peers { permanent_uuid: "21d9efbea7e94ce3b178138a3130a908" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36305 } } peers { permanent_uuid: "f6d90f1da9d041b3b9aa1e8d7da0657f" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40661 } }
I20260808 23:00:05.283835 31812 heartbeater.cc:503] Master 127.23.194.62:42601 was elected leader, sending a full tablet report...
I20260808 23:00:05.283983 32257 tablet_bootstrap.cc:654] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:05.283983 32252 raft_consensus.cc:348] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be361b70043347d2908f8fc817b7e841" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33157 } } peers { permanent_uuid: "21d9efbea7e94ce3b178138a3130a908" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36305 } } peers { permanent_uuid: "f6d90f1da9d041b3b9aa1e8d7da0657f" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40661 } }
I20260808 23:00:05.283829 32253 ts_tablet_manager.cc:1428] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 23:00:05.284150 32252 raft_consensus.cc:374] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:05.284188 32252 raft_consensus.cc:729] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be361b70043347d2908f8fc817b7e841, State: Initialized, Role: FOLLOWER
I20260808 23:00:05.284312 32257 log.cc:824] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:05.284294 32252 consensus_queue.cc:260] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be361b70043347d2908f8fc817b7e841" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33157 } } peers { permanent_uuid: "21d9efbea7e94ce3b178138a3130a908" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36305 } } peers { permanent_uuid: "f6d90f1da9d041b3b9aa1e8d7da0657f" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40661 } }
I20260808 23:00:05.284559 32252 ts_tablet_manager.cc:1428] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 23:00:05.284590 32230 heartbeater.cc:503] Master 127.23.194.62:42601 was elected leader, sending a full tablet report...
I20260808 23:00:05.285156 32257 tablet_bootstrap.cc:492] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f: No bootstrap required, opened a new log
I20260808 23:00:05.285223 32257 ts_tablet_manager.cc:1397] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:00:05.287034 32257 raft_consensus.cc:348] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be361b70043347d2908f8fc817b7e841" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33157 } } peers { permanent_uuid: "21d9efbea7e94ce3b178138a3130a908" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36305 } } peers { permanent_uuid: "f6d90f1da9d041b3b9aa1e8d7da0657f" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40661 } }
I20260808 23:00:05.287153 32257 raft_consensus.cc:374] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:05.287173 32257 raft_consensus.cc:729] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6d90f1da9d041b3b9aa1e8d7da0657f, State: Initialized, Role: FOLLOWER
I20260808 23:00:05.287249 32257 consensus_queue.cc:260] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be361b70043347d2908f8fc817b7e841" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33157 } } peers { permanent_uuid: "21d9efbea7e94ce3b178138a3130a908" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36305 } } peers { permanent_uuid: "f6d90f1da9d041b3b9aa1e8d7da0657f" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40661 } }
I20260808 23:00:05.287452 32257 ts_tablet_manager.cc:1428] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 23:00:05.287505 32021 heartbeater.cc:503] Master 127.23.194.62:42601 was elected leader, sending a full tablet report...
W20260808 23:00:05.389345 32022 tablet.cc:2369] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:00:05.510506 32231 tablet.cc:2369] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:00:05.527772 31813 tablet.cc:2369] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:00:05.573423 32260 raft_consensus.cc:482] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:00:05.573547 32260 raft_consensus.cc:504] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be361b70043347d2908f8fc817b7e841" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33157 } } peers { permanent_uuid: "21d9efbea7e94ce3b178138a3130a908" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36305 } } peers { permanent_uuid: "f6d90f1da9d041b3b9aa1e8d7da0657f" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40661 } }
I20260808 23:00:05.573895 32260 leader_election.cc:288] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be361b70043347d2908f8fc817b7e841 (127.23.194.3:33157), f6d90f1da9d041b3b9aa1e8d7da0657f (127.23.194.2:40661)
I20260808 23:00:05.575343 32247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.574204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:52061 (local address 127.23.194.3:33157)
0808 23:00:05.574257 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:05.574259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:05.574334 (+ 75us) server_negotiation.cc:407] Connection header received
0808 23:00:05.574388 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:05.574389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:05.574395 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:05.574466 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:05.575133 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:05.575135 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:05.575135 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:05.575139 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:05.575154 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:05.575158 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:05.575210 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:05.575259 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 23:00:05.575271 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 23:00:05.575343 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.574218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:37669 (local address 127.23.194.2:40661)
0808 23:00:05.574261 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:05.574263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:05.574334 (+ 71us) server_negotiation.cc:407] Connection header received
0808 23:00:05.574388 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:05.574389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:05.574395 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:05.574466 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:05.575085 (+ 619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:05.575088 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:05.575088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:05.575092 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:05.575112 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:05.575118 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:05.575198 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:05.575254 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 23:00:05.575268 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 23:00:05.575366 32266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.574097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:33157 (local address 127.23.194.1:52061)
0808 23:00:05.574262 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:05.574270 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:05.574331 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:05.574479 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:05.574480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:05.574495 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:05.575040 (+ 545us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:00:05.575041 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:05.575046 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:00:05.575054 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:05.575210 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:05.575211 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:05.575211 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 23:00:05.575250 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 23:00:05.575259 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":27,"thread_start_us":63,"threads_started":1}
I20260808 23:00:05.575371 32265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.574095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:40661 (local address 127.23.194.1:37669)
0808 23:00:05.574253 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:05.574265 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:05.574330 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:05.574495 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:05.574495 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:05.574496 (+ 1us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:05.574992 (+ 496us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:00:05.574992 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:05.575003 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:00:05.575019 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:05.575198 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:05.575199 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:05.575199 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 23:00:05.575244 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 23:00:05.575259 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260808 23:00:05.575757 32157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68c65243884e49a689e05750e410cf0b" candidate_uuid: "21d9efbea7e94ce3b178138a3130a908" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be361b70043347d2908f8fc817b7e841" is_pre_election: true
I20260808 23:00:05.575783 31944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68c65243884e49a689e05750e410cf0b" candidate_uuid: "21d9efbea7e94ce3b178138a3130a908" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6d90f1da9d041b3b9aa1e8d7da0657f" is_pre_election: true
I20260808 23:00:05.575919 31944 raft_consensus.cc:2454] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21d9efbea7e94ce3b178138a3130a908 in term 0.
I20260808 23:00:05.575919 32157 raft_consensus.cc:2454] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21d9efbea7e94ce3b178138a3130a908 in term 0.
I20260808 23:00:05.576175 31639 leader_election.cc:302] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21d9efbea7e94ce3b178138a3130a908, be361b70043347d2908f8fc817b7e841; no voters:
I20260808 23:00:05.576310 32260 raft_consensus.cc:2781] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:00:05.576372 32260 raft_consensus.cc:482] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:00:05.576388 32260 raft_consensus.cc:3037] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:05.577082 32260 raft_consensus.cc:504] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be361b70043347d2908f8fc817b7e841" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33157 } } peers { permanent_uuid: "21d9efbea7e94ce3b178138a3130a908" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36305 } } peers { permanent_uuid: "f6d90f1da9d041b3b9aa1e8d7da0657f" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40661 } }
I20260808 23:00:05.577211 32260 leader_election.cc:288] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908 [CANDIDATE]: Term 1 election: Requested vote from peers be361b70043347d2908f8fc817b7e841 (127.23.194.3:33157), f6d90f1da9d041b3b9aa1e8d7da0657f (127.23.194.2:40661)
I20260808 23:00:05.577360 32157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68c65243884e49a689e05750e410cf0b" candidate_uuid: "21d9efbea7e94ce3b178138a3130a908" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be361b70043347d2908f8fc817b7e841"
I20260808 23:00:05.577471 32157 raft_consensus.cc:3037] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:05.577522 31944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68c65243884e49a689e05750e410cf0b" candidate_uuid: "21d9efbea7e94ce3b178138a3130a908" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6d90f1da9d041b3b9aa1e8d7da0657f"
I20260808 23:00:05.577587 31944 raft_consensus.cc:3037] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:05.578111 32157 raft_consensus.cc:2454] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21d9efbea7e94ce3b178138a3130a908 in term 1.
I20260808 23:00:05.578238 31944 raft_consensus.cc:2454] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21d9efbea7e94ce3b178138a3130a908 in term 1.
I20260808 23:00:05.578291 31639 leader_election.cc:302] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21d9efbea7e94ce3b178138a3130a908, be361b70043347d2908f8fc817b7e841; no voters:
I20260808 23:00:05.578361 32260 raft_consensus.cc:2781] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:05.578492 32260 raft_consensus.cc:686] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908 [term 1 LEADER]: Becoming Leader. State: Replica: 21d9efbea7e94ce3b178138a3130a908, State: Running, Role: LEADER
I20260808 23:00:05.578588 32260 consensus_queue.cc:237] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be361b70043347d2908f8fc817b7e841" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33157 } } peers { permanent_uuid: "21d9efbea7e94ce3b178138a3130a908" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36305 } } peers { permanent_uuid: "f6d90f1da9d041b3b9aa1e8d7da0657f" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40661 } }
I20260808 23:00:05.579401 31520 catalog_manager.cc:5495] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908 reported cstate change: term changed from 0 to 1, leader changed from <none> to 21d9efbea7e94ce3b178138a3130a908 (127.23.194.1). New cstate: current_term: 1 leader_uuid: "21d9efbea7e94ce3b178138a3130a908" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be361b70043347d2908f8fc817b7e841" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21d9efbea7e94ce3b178138a3130a908" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6d90f1da9d041b3b9aa1e8d7da0657f" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40661 } health_report { overall_health: UNKNOWN } } }
I20260808 23:00:05.626694 32248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.625909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48558 (local address 127.23.194.1:36305)
0808 23:00:05.625970 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:05.625972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:05.625981 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:00:05.626018 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:05.626019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:05.626033 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:05.626081 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:05.626541 (+ 460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:05.626541 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:05.626541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:05.626543 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:05.626553 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:05.626556 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:05.626589 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:05.626622 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 23:00:05.626632 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 23:00:05.628496 31944 raft_consensus.cc:1261] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f [term 1 FOLLOWER]: Refusing update from remote peer 21d9efbea7e94ce3b178138a3130a908: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:00:05.628496 32157 raft_consensus.cc:1261] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841 [term 1 FOLLOWER]: Refusing update from remote peer 21d9efbea7e94ce3b178138a3130a908: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:00:05.628859 32260 consensus_queue.cc:1035] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6d90f1da9d041b3b9aa1e8d7da0657f" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:05.628997 32267 consensus_queue.cc:1035] T 68c65243884e49a689e05750e410cf0b P 21d9efbea7e94ce3b178138a3130a908 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be361b70043347d2908f8fc817b7e841" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:05.631269 32270 mvcc.cc:205] Tried to move back new op lower bound from 7316398103051694080 to 7316398102850596864. Current Snapshot: MvccSnapshot[applied={T|T < 7316398103051694080}]
I20260808 23:00:05.631639 32276 mvcc.cc:205] Tried to move back new op lower bound from 7316398103051694080 to 7316398102850596864. Current Snapshot: MvccSnapshot[applied={T|T < 7316398103051694080}]
I20260808 23:00:05.632292 32272 mvcc.cc:205] Tried to move back new op lower bound from 7316398103051694080 to 7316398102850596864. Current Snapshot: MvccSnapshot[applied={T|T < 7316398103051694080}]
I20260808 23:00:05.633800 31521 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:00:05.633893 31521 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:00:05.634653 31521 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dbf1299f7813423b923c1c358286e5c8: Sending DeleteTablet for 3 replicas of tablet 68c65243884e49a689e05750e410cf0b
I20260808 23:00:05.634970 32126 tablet_service.cc:1514] Processing DeleteTablet for tablet 68c65243884e49a689e05750e410cf0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:05 UTC) from {username='slave'} at 127.0.0.1:49228
I20260808 23:00:05.635088 31915 tablet_service.cc:1514] Processing DeleteTablet for tablet 68c65243884e49a689e05750e410cf0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:05 UTC) from {username='slave'} at 127.0.0.1:47134
I20260808 23:00:05.635197 31714 tablet_service.cc:1514] Processing DeleteTablet for tablet 68c65243884e49a689e05750e410cf0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:05 UTC) from {username='slave'} at 127.0.0.1:48546
I20260808 23:00:05.635281 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 31615
I20260808 23:00:05.640937 32287 tablet_replica.cc:330] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f: stopping tablet replica
I20260808 23:00:05.641031 32287 raft_consensus.cc:2229] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:00:05.641063 32286 tablet_replica.cc:330] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841: stopping tablet replica
I20260808 23:00:05.641125 32287 raft_consensus.cc:2258] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:05.641161 32286 raft_consensus.cc:2229] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:00:05.641251 32286 raft_consensus.cc:2258] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:05.641626 32287 ts_tablet_manager.cc:1905] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:05.642978 32286 ts_tablet_manager.cc:1905] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:05.643950 32287 ts_tablet_manager.cc:1918] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:00:05.644001 32287 log.cc:1196] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/68c65243884e49a689e05750e410cf0b
I20260808 23:00:05.644191 32287 ts_tablet_manager.cc:1939] T 68c65243884e49a689e05750e410cf0b P f6d90f1da9d041b3b9aa1e8d7da0657f: Deleting consensus metadata
I20260808 23:00:05.644518 32286 ts_tablet_manager.cc:1918] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:00:05.644555 31493 catalog_manager.cc:4841] TS f6d90f1da9d041b3b9aa1e8d7da0657f (127.23.194.2:40661): tablet 68c65243884e49a689e05750e410cf0b (table test-table [id=489a76e595bf49549afb6a595ebdc903]) successfully deleted
I20260808 23:00:05.644594 32286 log.cc:1196] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/68c65243884e49a689e05750e410cf0b
I20260808 23:00:05.644893 32286 ts_tablet_manager.cc:1939] T 68c65243884e49a689e05750e410cf0b P be361b70043347d2908f8fc817b7e841: Deleting consensus metadata
I20260808 23:00:05.645332 31495 catalog_manager.cc:4841] TS be361b70043347d2908f8fc817b7e841 (127.23.194.3:33157): tablet 68c65243884e49a689e05750e410cf0b (table test-table [id=489a76e595bf49549afb6a595ebdc903]) successfully deleted
W20260808 23:00:05.649297 31499 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.194.1:36305 (error 108)
I20260808 23:00:05.649761 32244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.649654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:36305 (local address 127.0.0.1:48568)
0808 23:00:05.649698 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:05.649733 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.1:36305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 23:00:05.649879 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 31818
W20260808 23:00:05.649880 31499 catalog_manager.cc:4584] TS 21d9efbea7e94ce3b178138a3130a908 (127.23.194.1:36305): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 68c65243884e49a689e05750e410cf0b: Network error: Client connection negotiation failed: client connection to 127.23.194.1:36305: connect: Connection refused (error 111)
I20260808 23:00:05.663128 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 32025
I20260808 23:00:05.671854 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 31466
2026-08-08T23:00:05Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (923 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 23:00:05.696005 24328 mini_oidc.cc:152] Starting JWKS server
I20260808 23:00:05.697247 24328 webserver.cc:460] Webserver started at https://127.0.0.1:40757/ using document root <none> and password file <none>
I20260808 23:00:05.697346 24328 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 23:00:05.697536 24328 webserver.cc:460] Webserver started at http://127.0.0.1:37521/ using document root <none> and password file <none>
2026-08-08T23:00:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:00:05Z Disabled control of system clock
I20260808 23:00:05.705709 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:33753
--webserver_interface=127.23.194.62
--webserver_port=0
--builtin_ntp_servers=127.23.194.20:39875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:33753
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40757/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229981209642-24328-0/testchainca.cert with env {}
W20260808 23:00:05.785820 32297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:05.786018 32297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:05.786039 32297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:05.787184 32297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:00:05.787225 32297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:05.787237 32297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:05.787259 32297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:00:05.787282 32297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 23:00:05.787305 32297 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 23:00:05.787326 32297 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40757/jwks.json
I20260808 23:00:05.788864 32297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:39875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:33753
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:33753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40757/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229981209642-24328-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:05.789103 32297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:05.789302 32297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:05.793900 32308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:05.793931 32305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:05.793994 32297 server_base.cc:1034] running on GCE node
W20260808 23:00:05.793905 32304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:05.794297 32297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:05.794590 32297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:05.795728 32297 hybrid_clock.cc:648] HybridClock initialized: now 1786230005795717 us; error 28 us; skew 500 ppm
I20260808 23:00:05.796844 32297 webserver.cc:460] Webserver started at http://127.23.194.62:37609/ using document root <none> and password file <none>
I20260808 23:00:05.797070 32297 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:05.797117 32297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:05.797240 32297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:05.798296 32297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "53c0afe6c90642bdbe9524af8868f1d1"
format_stamp: "Formatted at 2026-08-08 23:00:05 on dist-test-slave-f0tn"
server_key: "ff1ad643c077064a906f3854e19ee709"
server_key_iv: "a12d0fcf60c91e66acac00398ed68e81"
server_key_version: "encryptionkey@0"
I20260808 23:00:05.798620 32297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "53c0afe6c90642bdbe9524af8868f1d1"
format_stamp: "Formatted at 2026-08-08 23:00:05 on dist-test-slave-f0tn"
server_key: "ff1ad643c077064a906f3854e19ee709"
server_key_iv: "a12d0fcf60c91e66acac00398ed68e81"
server_key_version: "encryptionkey@0"
I20260808 23:00:05.799829 32297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:00:05.800446 32322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:05.800598 32297 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:05.800688 32297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "53c0afe6c90642bdbe9524af8868f1d1"
format_stamp: "Formatted at 2026-08-08 23:00:05 on dist-test-slave-f0tn"
server_key: "ff1ad643c077064a906f3854e19ee709"
server_key_iv: "a12d0fcf60c91e66acac00398ed68e81"
server_key_version: "encryptionkey@0"
I20260808 23:00:05.800753 32297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:05.819983 32297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:05.820299 32297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:05.820429 32297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:05.824312 32297 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:33753
I20260808 23:00:05.824348 32400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:33753 every 8 connection(s)
I20260808 23:00:05.824694 32297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:00:05.825661 32402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:05.827915 32402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1: Bootstrap starting.
I20260808 23:00:05.828486 32402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:05.828730 32402 log.cc:824] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:05.829370 32402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1: No bootstrap required, opened a new log
I20260808 23:00:05.829873 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 32297
I20260808 23:00:05.829972 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:00:05.830127 24328 external_mini_cluster.cc:1428] Setting key d530fc69ea5d2c60ba45127ecbb4cd23
I20260808 23:00:05.831339 32402 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53c0afe6c90642bdbe9524af8868f1d1" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 33753 } }
I20260808 23:00:05.831508 32402 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:05.831564 32402 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53c0afe6c90642bdbe9524af8868f1d1, State: Initialized, Role: FOLLOWER
I20260808 23:00:05.831701 32402 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53c0afe6c90642bdbe9524af8868f1d1" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 33753 } }
I20260808 23:00:05.831786 32402 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:00:05.831827 32402 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:00:05.831875 32402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:05.832551 32402 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53c0afe6c90642bdbe9524af8868f1d1" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 33753 } }
I20260808 23:00:05.832664 32402 leader_election.cc:302] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53c0afe6c90642bdbe9524af8868f1d1; no voters:
I20260808 23:00:05.832811 32402 leader_election.cc:288] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:00:05.832867 32414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:05.833050 32402 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:00:05.833056 32414 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1 [term 1 LEADER]: Becoming Leader. State: Replica: 53c0afe6c90642bdbe9524af8868f1d1, State: Running, Role: LEADER
I20260808 23:00:05.833186 32414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53c0afe6c90642bdbe9524af8868f1d1" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 33753 } }
I20260808 23:00:05.833662 32417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53c0afe6c90642bdbe9524af8868f1d1. Latest consensus state: current_term: 1 leader_uuid: "53c0afe6c90642bdbe9524af8868f1d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53c0afe6c90642bdbe9524af8868f1d1" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 33753 } } }
I20260808 23:00:05.833806 32417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:05.834161 32415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53c0afe6c90642bdbe9524af8868f1d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53c0afe6c90642bdbe9524af8868f1d1" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 33753 } } }
I20260808 23:00:05.834300 32415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:05.834214 32430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:00:05.835916 32430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:00:05.837128 32412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.830613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40050 (local address 127.23.194.62:33753)
0808 23:00:05.830841 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:05.830850 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:05.830885 (+ 35us) server_negotiation.cc:407] Connection header received
0808 23:00:05.831312 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:05.831322 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:05.831410 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:05.831554 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:05.831757 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:05.833519 (+ 1762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:05.834610 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:05.835110 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:05.835239 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:05.835983 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:05.835993 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:05.835994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:05.836249 (+ 255us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:05.836299 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:05.836314 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:05.836384 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:05.836821 (+ 437us) server_negotiation.cc:299] Negotiation successful
0808 23:00:05.836893 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
I20260808 23:00:05.837560 32430 catalog_manager.cc:1347] Generated new cluster ID: 14eb5fd0d97246288a68aa2003624611
I20260808 23:00:05.837611 32430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:00:05.837981 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:00:05.844805 32430 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:00:05.845295 32430 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:00:05.849798 32430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53c0afe6c90642bdbe9524af8868f1d1: Generated new TSK 0
I20260808 23:00:05.849964 32430 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 23:00:05.874480 32446 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 23:00:05.874630 32446 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260808 23:00:05.875156 32445 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:05.860043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:33753 (local address 127.0.0.1:40056)
0808 23:00:05.860245 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:05.860255 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:05.860320 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:05.860482 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:05.860483 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:05.860485 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 23:00:05.860555 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:05.860557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:05.861174 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:05.861176 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:05.861850 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:05.861853 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:05.862185 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:05.862186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:05.862250 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:05.862270 (+ 20us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 23:00:05.874742 (+ 12472us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40757/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 23:00:05.874840 (+ 98us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.194.62:33753: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40757/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.23.194.62:33753: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40757/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 23:00:05.875435 32412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:05.860081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40056 (local address 127.23.194.62:33753)
0808 23:00:05.860136 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:05.860138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:05.860311 (+ 173us) server_negotiation.cc:407] Connection header received
0808 23:00:05.860403 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:05.860404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:05.860417 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:05.860471 (+ 54us) server_negotiation.cc:226] Negotiated authn=JWT
0808 23:00:05.860610 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:05.861100 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:05.861927 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:05.862111 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:05.862164 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:05.862371 (+ 207us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 23:00:05.874576 (+ 12205us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40757/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 23:00:05.875375 (+ 799us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40056: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40757/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 23:00:05.875463 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 32297
W20260808 23:00:05.875484 32412 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40056: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40757/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
2026-08-08T23:00:05Z chronyd exiting
I20260808 23:00:05.932821 24328 test_util.cc:175] -----------------------------------------------
I20260808 23:00:05.932904 24328 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786229981209642-24328-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (238 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 23:00:05.933718 24328 mini_oidc.cc:152] Starting JWKS server
I20260808 23:00:05.934369 24328 webserver.cc:460] Webserver started at https://127.0.0.1:41325/ using document root <none> and password file <none>
I20260808 23:00:05.934408 24328 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 23:00:05.934514 24328 webserver.cc:460] Webserver started at http://127.0.0.1:34795/ using document root <none> and password file <none>
2026-08-08T23:00:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:00:05Z Disabled control of system clock
I20260808 23:00:05.943193 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:38755
--webserver_interface=127.23.194.62
--webserver_port=0
--builtin_ntp_servers=127.23.194.20:39479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:38755
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41325/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229981209642-24328-0/testchainca.cert with env {}
W20260808 23:00:06.022912 32455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:06.023116 32455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:06.023145 32455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:06.024247 32455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:00:06.024281 32455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:06.024291 32455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:06.024302 32455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:00:06.024313 32455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 23:00:06.024323 32455 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 23:00:06.024334 32455 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41325/jwks.json
I20260808 23:00:06.025870 32455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:39479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:38755
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:38755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41325/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229981209642-24328-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:06.026079 32455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:06.026254 32455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:06.030961 32467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:06.030989 32463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:06.031021 32464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:06.031006 32455 server_base.cc:1034] running on GCE node
I20260808 23:00:06.031388 32455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:06.031656 32455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:06.032804 32455 hybrid_clock.cc:648] HybridClock initialized: now 1786230006032794 us; error 34 us; skew 500 ppm
I20260808 23:00:06.034050 32455 webserver.cc:460] Webserver started at http://127.23.194.62:34681/ using document root <none> and password file <none>
I20260808 23:00:06.034269 32455 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:06.034312 32455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:06.034400 32455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:06.035411 32455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "6a2a399994aa40acad9571b4b58796bb"
format_stamp: "Formatted at 2026-08-08 23:00:06 on dist-test-slave-f0tn"
server_key: "c399e0ff6e5ee357192d410081620e02"
server_key_iv: "7b17b0fc3f7a3bf64c17e87c3617f516"
server_key_version: "encryptionkey@0"
I20260808 23:00:06.035705 32455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "6a2a399994aa40acad9571b4b58796bb"
format_stamp: "Formatted at 2026-08-08 23:00:06 on dist-test-slave-f0tn"
server_key: "c399e0ff6e5ee357192d410081620e02"
server_key_iv: "7b17b0fc3f7a3bf64c17e87c3617f516"
server_key_version: "encryptionkey@0"
I20260808 23:00:06.036855 32455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 23:00:06.037597 32473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:06.037746 32455 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:06.037833 32455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "6a2a399994aa40acad9571b4b58796bb"
format_stamp: "Formatted at 2026-08-08 23:00:06 on dist-test-slave-f0tn"
server_key: "c399e0ff6e5ee357192d410081620e02"
server_key_iv: "7b17b0fc3f7a3bf64c17e87c3617f516"
server_key_version: "encryptionkey@0"
I20260808 23:00:06.037892 32455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:06.080374 32455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:06.080713 32455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:06.080857 32455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:06.084817 32455 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:38755
I20260808 23:00:06.084863 32568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:38755 every 8 connection(s)
I20260808 23:00:06.085210 32455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:00:06.085929 32569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:06.087630 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 32455
I20260808 23:00:06.087747 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:00:06.087929 24328 external_mini_cluster.cc:1428] Setting key e9b3cad54474c97d33076b2aab482428
I20260808 23:00:06.089300 32569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb: Bootstrap starting.
I20260808 23:00:06.089965 32569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:06.090227 32569 log.cc:824] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:06.090878 32569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb: No bootstrap required, opened a new log
I20260808 23:00:06.092749 32569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a2a399994aa40acad9571b4b58796bb" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 38755 } }
I20260808 23:00:06.092892 32569 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:06.092926 32569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a2a399994aa40acad9571b4b58796bb, State: Initialized, Role: FOLLOWER
I20260808 23:00:06.093093 32569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a2a399994aa40acad9571b4b58796bb" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 38755 } }
I20260808 23:00:06.093194 32569 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:00:06.093245 32569 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:00:06.093312 32569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:06.093417 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:06.088476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42854 (local address 127.23.194.62:38755)
0808 23:00:06.088695 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:06.088703 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:06.088729 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:00:06.089147 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:06.089155 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:06.089217 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:06.089356 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:06.089608 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:06.090570 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:06.091380 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:06.091805 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:06.091919 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:06.092651 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:06.092655 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:06.092655 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:06.092823 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:06.092861 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:06.092871 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:06.092928 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:06.093051 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 23:00:06.093136 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":87,"threads_started":1}
I20260808 23:00:06.094050 32569 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a2a399994aa40acad9571b4b58796bb" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 38755 } }
I20260808 23:00:06.094166 32569 leader_election.cc:302] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a2a399994aa40acad9571b4b58796bb; no voters:
I20260808 23:00:06.094345 32569 leader_election.cc:288] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:00:06.094372 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:00:06.094384 32576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:06.094530 32576 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb [term 1 LEADER]: Becoming Leader. State: Replica: 6a2a399994aa40acad9571b4b58796bb, State: Running, Role: LEADER
I20260808 23:00:06.094621 32569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:00:06.094615 32576 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a2a399994aa40acad9571b4b58796bb" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 38755 } }
I20260808 23:00:06.095031 32579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a2a399994aa40acad9571b4b58796bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a2a399994aa40acad9571b4b58796bb" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 38755 } } }
I20260808 23:00:06.095113 32579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:06.095286 32577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a2a399994aa40acad9571b4b58796bb. Latest consensus state: current_term: 1 leader_uuid: "6a2a399994aa40acad9571b4b58796bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a2a399994aa40acad9571b4b58796bb" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 38755 } } }
I20260808 23:00:06.095350 32577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:06.095491 32587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:00:06.096024 32587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:00:06.097517 32587 catalog_manager.cc:1347] Generated new cluster ID: 5e17c7aaa041445c961a28eceb67787e
I20260808 23:00:06.097574 32587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:00:06.124255 32587 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:00:06.124805 32587 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:00:06.128789 32587 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a2a399994aa40acad9571b4b58796bb: Generated new TSK 0
I20260808 23:00:06.129010 32587 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 23:00:06.155639 32622 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 23:00:06.155735 32622 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 23:00:06.156036 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:06.134139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42862 (local address 127.23.194.62:38755)
0808 23:00:06.134186 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:06.134188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:06.134270 (+ 82us) server_negotiation.cc:407] Connection header received
0808 23:00:06.134349 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:06.134350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:06.134359 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:06.134417 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0808 23:00:06.134550 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:06.135120 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:06.136231 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:06.136420 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:06.136462 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:06.136630 (+ 168us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 23:00:06.155810 (+ 19180us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 23:00:06.155966 (+ 156us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42862: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 23:00:06.156091 32573 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42862: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 23:00:06.156174 32621 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:06.134050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:38755 (local address 127.0.0.1:42862)
0808 23:00:06.134198 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:06.134209 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:06.134280 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:06.134427 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:06.134428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:06.134430 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 23:00:06.134497 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:06.134500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:06.135197 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:06.135198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:06.136159 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:06.136163 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:06.136481 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:06.136482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:06.136543 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:06.136549 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 23:00:06.155939 (+ 19390us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 23:00:06.156066 (+ 127us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.194.62:38755: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":95,"threads_started":1}
I20260808 23:00:06.156456 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 32455
2026-08-08T23:00:06Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (267 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 23:00:06.201763 24328 mini_oidc.cc:152] Starting JWKS server
I20260808 23:00:06.202427 24328 webserver.cc:460] Webserver started at https://127.0.0.1:43303/ using document root <none> and password file <none>
I20260808 23:00:06.202476 24328 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 23:00:06.202610 24328 webserver.cc:460] Webserver started at http://127.0.0.1:39439/ using document root <none> and password file <none>
2026-08-08T23:00:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:00:06Z Disabled control of system clock
I20260808 23:00:06.211059 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:45657
--webserver_interface=127.23.194.62
--webserver_port=0
--builtin_ntp_servers=127.23.194.20:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:45657
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43303/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 23:00:06.292846 32632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:06.293061 32632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:06.293081 32632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:06.294252 32632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:00:06.294296 32632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:06.294307 32632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:06.294318 32632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:00:06.294329 32632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 23:00:06.294343 32632 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 23:00:06.294354 32632 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43303/jwks.json
I20260808 23:00:06.295738 32632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:45657
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:45657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43303/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:06.295945 32632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:06.296192 32632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:06.300865 32638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:06.300891 32641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:06.300899 32639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:06.301133 32632 server_base.cc:1034] running on GCE node
I20260808 23:00:06.301352 32632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:06.301627 32632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:06.302767 32632 hybrid_clock.cc:648] HybridClock initialized: now 1786230006302754 us; error 30 us; skew 500 ppm
I20260808 23:00:06.303859 32632 webserver.cc:460] Webserver started at http://127.23.194.62:33073/ using document root <none> and password file <none>
I20260808 23:00:06.304064 32632 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:06.304105 32632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:06.304212 32632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:06.305212 32632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "7a376402b58843cdafa4c3ba9e423d93"
format_stamp: "Formatted at 2026-08-08 23:00:06 on dist-test-slave-f0tn"
server_key: "9209d349d88aa3d83cde5114f2245493"
server_key_iv: "58ce9b388191392059ce1a659f826bbe"
server_key_version: "encryptionkey@0"
I20260808 23:00:06.305538 32632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "7a376402b58843cdafa4c3ba9e423d93"
format_stamp: "Formatted at 2026-08-08 23:00:06 on dist-test-slave-f0tn"
server_key: "9209d349d88aa3d83cde5114f2245493"
server_key_iv: "58ce9b388191392059ce1a659f826bbe"
server_key_version: "encryptionkey@0"
I20260808 23:00:06.306659 32632 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 23:00:06.307276 32650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:06.307418 32632 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:00:06.307493 32632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "7a376402b58843cdafa4c3ba9e423d93"
format_stamp: "Formatted at 2026-08-08 23:00:06 on dist-test-slave-f0tn"
server_key: "9209d349d88aa3d83cde5114f2245493"
server_key_iv: "58ce9b388191392059ce1a659f826bbe"
server_key_version: "encryptionkey@0"
I20260808 23:00:06.307559 32632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:06.337636 32632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:06.337993 32632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:06.338133 32632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:06.341873 32632 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:45657
I20260808 23:00:06.341917 32728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:45657 every 8 connection(s)
I20260808 23:00:06.342247 32632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:00:06.342938 32730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:06.345731 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 32632
I20260808 23:00:06.345755 32730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93: Bootstrap starting.
I20260808 23:00:06.345841 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:00:06.346021 24328 external_mini_cluster.cc:1428] Setting key b823f963f2a089f216f47b3ed80e7eb9
I20260808 23:00:06.346398 32730 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:06.346696 32730 log.cc:824] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:06.347411 32730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93: No bootstrap required, opened a new log
I20260808 23:00:06.349241 32730 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a376402b58843cdafa4c3ba9e423d93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 45657 } }
I20260808 23:00:06.349409 32730 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:06.349463 32730 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a376402b58843cdafa4c3ba9e423d93, State: Initialized, Role: FOLLOWER
I20260808 23:00:06.349596 32730 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a376402b58843cdafa4c3ba9e423d93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 45657 } }
I20260808 23:00:06.349669 32730 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:00:06.349712 32730 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:00:06.349763 32730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:06.350320 32730 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a376402b58843cdafa4c3ba9e423d93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 45657 } }
I20260808 23:00:06.350425 32730 leader_election.cc:302] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a376402b58843cdafa4c3ba9e423d93; no voters:
I20260808 23:00:06.350605 32730 leader_election.cc:288] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:00:06.350692 32742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:06.350903 32742 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93 [term 1 LEADER]: Becoming Leader. State: Replica: 7a376402b58843cdafa4c3ba9e423d93, State: Running, Role: LEADER
I20260808 23:00:06.350896 32730 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:00:06.351089 32742 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a376402b58843cdafa4c3ba9e423d93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 45657 } }
I20260808 23:00:06.351487 32743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a376402b58843cdafa4c3ba9e423d93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a376402b58843cdafa4c3ba9e423d93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 45657 } } }
I20260808 23:00:06.351532 32740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:06.346538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52724 (local address 127.23.194.62:45657)
0808 23:00:06.346762 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:06.346771 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:06.346805 (+ 34us) server_negotiation.cc:407] Connection header received
0808 23:00:06.347254 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:06.347260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:06.347317 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:06.347447 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:06.347668 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:06.348561 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:06.349324 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:06.349918 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:06.350035 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:06.350701 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:06.350705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:06.350706 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:06.350952 (+ 246us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:06.350997 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:06.351025 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:06.351083 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:06.351159 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 23:00:06.351329 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260808 23:00:06.351595 32743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:06.351747 32744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a376402b58843cdafa4c3ba9e423d93. Latest consensus state: current_term: 1 leader_uuid: "7a376402b58843cdafa4c3ba9e423d93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a376402b58843cdafa4c3ba9e423d93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 45657 } } }
I20260808 23:00:06.351819 32744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:06.352149 32753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:00:06.352536 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:00:06.352622 32753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:00:06.354234 32753 catalog_manager.cc:1347] Generated new cluster ID: f68a415ebc46442ba526687d5ba1203c
I20260808 23:00:06.354295 32753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:00:06.360668 32753 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:00:06.361155 32753 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:00:06.363286 32753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a376402b58843cdafa4c3ba9e423d93: Generated new TSK 0
I20260808 23:00:06.363461 32753 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 23:00:06.373065 311 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:06.369135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:45657 (local address 127.0.0.1:52726)
0808 23:00:06.369293 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:06.369304 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:06.369372 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:06.369566 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:06.369567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:06.369569 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 23:00:06.369635 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:06.369637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:06.370315 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:06.370316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:06.371269 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:06.371272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:06.371634 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:06.371635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:06.371703 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:06.371709 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 23:00:06.372851 (+ 1142us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43303/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 23:00:06.372958 (+ 107us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.194.62:45657: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43303/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":108,"threads_started":1}
I20260808 23:00:06.373327 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 32632
E20260808 23:00:06.403685 312 webserver.cc:533] Webserver: sslize failed:
E20260808 23:00:06.403769 312 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T23:00:06Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (229 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T23:00:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:00:06Z Disabled control of system clock
I20260808 23:00:06.439417 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:41909
--webserver_interface=127.23.194.62
--webserver_port=0
--builtin_ntp_servers=127.23.194.20:46049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:41909
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229981209642-24328-0/testchainca.cert with env {}
W20260808 23:00:06.520351 317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:06.520548 317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:06.520570 317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:06.521826 317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:00:06.521868 317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:06.521880 317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:06.521903 317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:00:06.521926 317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 23:00:06.521950 317 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 23:00:06.521971 317 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 23:00:06.523481 317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:46049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:41909
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:41909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229981209642-24328-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:06.523717 317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:06.523912 317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:06.528527 323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:06.528529 324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:06.528564 326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:06.528885 317 server_base.cc:1034] running on GCE node
I20260808 23:00:06.529088 317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:06.529345 317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:06.530515 317 hybrid_clock.cc:648] HybridClock initialized: now 1786230006530498 us; error 30 us; skew 500 ppm
I20260808 23:00:06.531711 317 webserver.cc:460] Webserver started at http://127.23.194.62:33845/ using document root <none> and password file <none>
I20260808 23:00:06.531940 317 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:06.531987 317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:06.532088 317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:06.533104 317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "5b8619a0576649a79b69dbb870f4ada5"
format_stamp: "Formatted at 2026-08-08 23:00:06 on dist-test-slave-f0tn"
server_key: "6ebfa6ac5e570f7af939c6e325de6e5b"
server_key_iv: "0a8b003fb8bad16963755e0af90b8205"
server_key_version: "encryptionkey@0"
I20260808 23:00:06.533468 317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "5b8619a0576649a79b69dbb870f4ada5"
format_stamp: "Formatted at 2026-08-08 23:00:06 on dist-test-slave-f0tn"
server_key: "6ebfa6ac5e570f7af939c6e325de6e5b"
server_key_iv: "0a8b003fb8bad16963755e0af90b8205"
server_key_version: "encryptionkey@0"
I20260808 23:00:06.534715 317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:00:06.535419 336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:06.535570 317 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:06.535662 317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "5b8619a0576649a79b69dbb870f4ada5"
format_stamp: "Formatted at 2026-08-08 23:00:06 on dist-test-slave-f0tn"
server_key: "6ebfa6ac5e570f7af939c6e325de6e5b"
server_key_iv: "0a8b003fb8bad16963755e0af90b8205"
server_key_version: "encryptionkey@0"
I20260808 23:00:06.535725 317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:06.542343 317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:06.542590 317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:06.542712 317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:06.546610 317 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:41909
I20260808 23:00:06.546644 424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:41909 every 8 connection(s)
I20260808 23:00:06.546972 317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:00:06.547950 426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:06.550173 426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5: Bootstrap starting.
I20260808 23:00:06.550700 426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:06.550922 426 log.cc:824] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:06.551519 426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5: No bootstrap required, opened a new log
I20260808 23:00:06.553046 426 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b8619a0576649a79b69dbb870f4ada5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 41909 } }
I20260808 23:00:06.553177 426 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:06.553206 426 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b8619a0576649a79b69dbb870f4ada5, State: Initialized, Role: FOLLOWER
I20260808 23:00:06.553332 426 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b8619a0576649a79b69dbb870f4ada5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 41909 } }
I20260808 23:00:06.553436 426 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:00:06.553474 426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:00:06.553471 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 317
I20260808 23:00:06.553522 426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:06.553565 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:00:06.553761 24328 external_mini_cluster.cc:1428] Setting key 44958c86747d2550d313ecc90ff44471
I20260808 23:00:06.554149 426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b8619a0576649a79b69dbb870f4ada5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 41909 } }
I20260808 23:00:06.554256 426 leader_election.cc:302] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b8619a0576649a79b69dbb870f4ada5; no voters:
I20260808 23:00:06.554406 426 leader_election.cc:288] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:00:06.554450 434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:06.554570 434 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5 [term 1 LEADER]: Becoming Leader. State: Replica: 5b8619a0576649a79b69dbb870f4ada5, State: Running, Role: LEADER
I20260808 23:00:06.554664 434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b8619a0576649a79b69dbb870f4ada5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 41909 } }
I20260808 23:00:06.554668 426 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:00:06.555007 437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b8619a0576649a79b69dbb870f4ada5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b8619a0576649a79b69dbb870f4ada5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 41909 } } }
I20260808 23:00:06.555090 437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:06.555335 438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b8619a0576649a79b69dbb870f4ada5. Latest consensus state: current_term: 1 leader_uuid: "5b8619a0576649a79b69dbb870f4ada5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b8619a0576649a79b69dbb870f4ada5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 41909 } } }
I20260808 23:00:06.555398 438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:06.555864 443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:00:06.556440 443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:00:06.558542 443 catalog_manager.cc:1347] Generated new cluster ID: dbe0a17b09de4787ad5378deaaf043ab
I20260808 23:00:06.558596 443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:00:06.559218 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:06.554230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56862 (local address 127.23.194.62:41909)
0808 23:00:06.554425 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:06.554433 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:06.554467 (+ 34us) server_negotiation.cc:407] Connection header received
0808 23:00:06.554583 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:06.554590 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:06.554661 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:06.555089 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:06.555198 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:06.556533 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:06.557268 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:06.557836 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:06.557956 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:06.558614 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:06.558617 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:06.558618 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:06.558789 (+ 171us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:06.558822 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:06.558833 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:06.558886 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:06.558962 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 23:00:06.559037 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
I20260808 23:00:06.560076 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:00:06.569109 443 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:00:06.569625 443 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:00:06.578948 443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b8619a0576649a79b69dbb870f4ada5: Generated new TSK 0
I20260808 23:00:06.579099 443 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:00:06.605405 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:06.589349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56864 (local address 127.23.194.62:41909)
0808 23:00:06.589426 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:06.589428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:06.589524 (+ 96us) server_negotiation.cc:407] Connection header received
0808 23:00:06.589606 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:06.589607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:06.589619 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:06.589684 (+ 65us) server_negotiation.cc:226] Negotiated authn=JWT
0808 23:00:06.589825 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:06.590352 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:06.591454 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:06.591681 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:06.591745 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:06.592036 (+ 291us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 23:00:06.605134 (+ 13098us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 23:00:06.605315 (+ 181us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56864: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 23:00:06.605496 433 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56864: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 23:00:06.605597 470 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:06.589252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:41909 (local address 127.0.0.1:56864)
0808 23:00:06.589447 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:06.589458 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:06.589534 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:06.589696 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:06.589696 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:06.589699 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0808 23:00:06.589767 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:06.589770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:06.590426 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:06.590427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:06.591343 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:06.591347 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:06.591765 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:06.591766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:06.591827 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:06.591834 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 23:00:06.605297 (+ 13463us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 23:00:06.605478 (+ 181us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.194.62:41909: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":102,"threads_started":1}
I20260808 23:00:06.605922 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 317
2026-08-08T23:00:06Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (204 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (90 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (87 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:00:06 dist-test-slave-f0tn krb5kdc[485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:00:06 dist-test-slave-f0tn krb5kdc[485](info): set up 2 sockets
Aug 08 23:00:06 dist-test-slave-f0tn krb5kdc[485](info): commencing operation
krb5kdc: starting...
W20260808 23:00:07.943770 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:00:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:00:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:00:08.060902 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:32989
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:41579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:32989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:08.144198 509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:08.144434 509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:08.144469 509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:08.145714 509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:00:08.145766 509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:08.145789 509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:08.145808 509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:00:08.145829 509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:00:08.147390 509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:41579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:32989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:32989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:08.147660 509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:08.147883 509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 23:00:08.152859 509 server_base.cc:1034] running on GCE node
W20260808 23:00:08.152827 516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:08.152792 515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:08.152814 521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:08.153266 509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:08.153576 509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:08.154719 509 hybrid_clock.cc:648] HybridClock initialized: now 1786230008154705 us; error 29 us; skew 500 ppm
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230008, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:08.157078 509 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:00:08.157537 509 webserver.cc:460] Webserver started at http://127.23.194.62:40307/ using document root <none> and password file <none>
I20260808 23:00:08.157733 509 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:08.157773 509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:08.157866 509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:08.158857 509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "c37d3452b9154891a9ec6c03cf89f2a4"
format_stamp: "Formatted at 2026-08-08 23:00:08 on dist-test-slave-f0tn"
server_key: "5f199cf3fec75240e0e5c7bd423be644"
server_key_iv: "d66795423b7b36b104ff9c29679b22f9"
server_key_version: "encryptionkey@0"
I20260808 23:00:08.159144 509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "c37d3452b9154891a9ec6c03cf89f2a4"
format_stamp: "Formatted at 2026-08-08 23:00:08 on dist-test-slave-f0tn"
server_key: "5f199cf3fec75240e0e5c7bd423be644"
server_key_iv: "d66795423b7b36b104ff9c29679b22f9"
server_key_version: "encryptionkey@0"
I20260808 23:00:08.160338 509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:00:08.160974 532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:08.161132 509 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:08.161233 509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "c37d3452b9154891a9ec6c03cf89f2a4"
format_stamp: "Formatted at 2026-08-08 23:00:08 on dist-test-slave-f0tn"
server_key: "5f199cf3fec75240e0e5c7bd423be644"
server_key_iv: "d66795423b7b36b104ff9c29679b22f9"
server_key_version: "encryptionkey@0"
I20260808 23:00:08.161289 509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:08.179503 509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:08.181244 509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:08.181355 509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:08.185021 509 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:32989
I20260808 23:00:08.185068 616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:32989 every 8 connection(s)
I20260808 23:00:08.185428 509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:00:08.185683 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 509
I20260808 23:00:08.185796 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:00:08.186017 24328 external_mini_cluster.cc:1428] Setting key 7533b6d9d4ed786acacfed976811cc6e
I20260808 23:00:08.186492 619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:08.189595 619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4: Bootstrap starting.
I20260808 23:00:08.190209 619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:08.190497 619 log.cc:824] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:08.191254 619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4: No bootstrap required, opened a new log
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:08.193113 619 raft_consensus.cc:348] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37d3452b9154891a9ec6c03cf89f2a4" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 32989 } }
I20260808 23:00:08.193285 619 raft_consensus.cc:374] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:08.193326 619 raft_consensus.cc:729] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37d3452b9154891a9ec6c03cf89f2a4, State: Initialized, Role: FOLLOWER
I20260808 23:00:08.193480 619 consensus_queue.cc:260] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37d3452b9154891a9ec6c03cf89f2a4" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 32989 } }
I20260808 23:00:08.193547 619 raft_consensus.cc:388] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:00:08.193573 619 raft_consensus.cc:482] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:00:08.193619 619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:08.194278 619 raft_consensus.cc:504] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37d3452b9154891a9ec6c03cf89f2a4" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 32989 } }
I20260808 23:00:08.194387 619 leader_election.cc:302] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c37d3452b9154891a9ec6c03cf89f2a4; no voters:
I20260808 23:00:08.194545 619 leader_election.cc:288] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:00:08.194597 632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:08.194803 632 raft_consensus.cc:686] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4 [term 1 LEADER]: Becoming Leader. State: Replica: c37d3452b9154891a9ec6c03cf89f2a4, State: Running, Role: LEADER
I20260808 23:00:08.194811 619 sys_catalog.cc:564] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:00:08.194968 632 consensus_queue.cc:237] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37d3452b9154891a9ec6c03cf89f2a4" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 32989 } }
I20260808 23:00:08.195523 633 sys_catalog.cc:455] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c37d3452b9154891a9ec6c03cf89f2a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37d3452b9154891a9ec6c03cf89f2a4" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 32989 } } }
I20260808 23:00:08.195605 633 sys_catalog.cc:458] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:08.195803 648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:00:08.195806 634 sys_catalog.cc:455] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c37d3452b9154891a9ec6c03cf89f2a4. Latest consensus state: current_term: 1 leader_uuid: "c37d3452b9154891a9ec6c03cf89f2a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37d3452b9154891a9ec6c03cf89f2a4" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 32989 } } }
I20260808 23:00:08.195950 634 sys_catalog.cc:458] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:08.196422 648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:00:08.196959 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:08.186523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39148 (local address 127.23.194.62:32989)
0808 23:00:08.186712 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:08.186721 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:08.186750 (+ 29us) server_negotiation.cc:407] Connection header received
0808 23:00:08.187230 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:08.187239 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:08.187301 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:08.187447 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:08.188042 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:08.189412 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:08.190147 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:08.190745 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:08.190829 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:08.193005 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:08.193010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:08.193010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:08.193023 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:08.195015 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:08.195456 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:08.195459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:08.195459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:08.195566 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:08.195791 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:08.195792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:08.195792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:08.196117 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:08.196284 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:08.196627 (+ 343us) server_negotiation.cc:299] Negotiation successful
0808 23:00:08.196765 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260808 23:00:08.198261 648 catalog_manager.cc:1347] Generated new cluster ID: ce6cfe4245ee4edf9243a8ab03053bd0
I20260808 23:00:08.198314 648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:00:08.209180 648 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:00:08.209919 648 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:00:08.213258 648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c37d3452b9154891a9ec6c03cf89f2a4: Generated new TSK 0
I20260808 23:00:08.213469 648 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
I20260808 23:00:08.237876 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:32989
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:41579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:08.320029 669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:08.320242 669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:08.320266 669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:08.321486 669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:08.321529 669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:08.321556 669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:00:08.323181 669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:41579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:32989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:08.323427 669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:08.323655 669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:08.328698 679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:08.328790 669 server_base.cc:1034] running on GCE node
W20260808 23:00:08.328677 685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:08.328830 680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:08.329116 669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:08.329350 669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:08.330502 669 hybrid_clock.cc:648] HybridClock initialized: now 1786230008330491 us; error 28 us; skew 500 ppm
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230008, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:08.333016 669 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 23:00:08.333491 669 webserver.cc:460] Webserver started at http://127.23.194.1:43165/ using document root <none> and password file <none>
I20260808 23:00:08.333739 669 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:08.333791 669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:08.333913 669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:08.334951 669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "fb5e5477f5cb4c228b9a72e3aea99d59"
format_stamp: "Formatted at 2026-08-08 23:00:08 on dist-test-slave-f0tn"
server_key: "edd87eaa503d0e44238d22a25e073d31"
server_key_iv: "caa18da484e46d3f1ab90ca65f855743"
server_key_version: "encryptionkey@0"
I20260808 23:00:08.335275 669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "fb5e5477f5cb4c228b9a72e3aea99d59"
format_stamp: "Formatted at 2026-08-08 23:00:08 on dist-test-slave-f0tn"
server_key: "edd87eaa503d0e44238d22a25e073d31"
server_key_iv: "caa18da484e46d3f1ab90ca65f855743"
server_key_version: "encryptionkey@0"
I20260808 23:00:08.336506 669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 23:00:08.337154 695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:08.337306 669 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:08.337412 669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "fb5e5477f5cb4c228b9a72e3aea99d59"
format_stamp: "Formatted at 2026-08-08 23:00:08 on dist-test-slave-f0tn"
server_key: "edd87eaa503d0e44238d22a25e073d31"
server_key_iv: "caa18da484e46d3f1ab90ca65f855743"
server_key_version: "encryptionkey@0"
I20260808 23:00:08.337476 669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:08.348884 669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:08.351054 669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:08.351193 669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:08.351401 669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:08.351701 669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:08.351737 669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:08.351763 669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:08.351790 669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:08.357355 669 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:32781
I20260808 23:00:08.357434 880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:32781 every 8 connection(s)
I20260808 23:00:08.357718 669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:00:08.362874 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 669
I20260808 23:00:08.362982 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:00:08.363189 24328 external_mini_cluster.cc:1428] Setting key c7f254807a17246e09a70888742d171b
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230008, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:08.366858 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:08.358462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:60621 (local address 127.23.194.62:32989)
0808 23:00:08.358550 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:08.358552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:08.358791 (+ 239us) server_negotiation.cc:407] Connection header received
0808 23:00:08.359699 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:08.359700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:08.359710 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:08.359758 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:08.360301 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:08.360796 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:08.361776 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:08.361965 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:08.362013 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:08.364069 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:08.364071 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:08.364071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:08.364079 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:08.365560 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:08.365998 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:08.365999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:08.366000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:08.366050 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:08.366302 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:08.366302 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:08.366303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:08.366395 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:08.366439 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:08.366723 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 23:00:08.366792 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 23:00:08.367121 883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:08.358439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:32989 (local address 127.23.194.1:60621)
0808 23:00:08.358636 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:08.358683 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:08.359610 (+ 927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:08.359812 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:08.359815 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:08.360045 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:08.360237 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:08.360241 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:08.360855 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:08.360855 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:08.361700 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:08.361702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:08.362019 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:08.362020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:08.362115 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:08.362514 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:08.362521 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:08.363979 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:08.365630 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:08.365637 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:08.365639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:08.365929 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:08.366104 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:08.366104 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:08.366105 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:08.366246 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:08.366443 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:08.366448 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:08.366600 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 23:00:08.366710 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 23:00:08.366875 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260808 23:00:08.367604 881 heartbeater.cc:347] Connected to a master server at 127.23.194.62:32989
I20260808 23:00:08.367697 881 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:08.367857 881 heartbeater.cc:511] Master 127.23.194.62:32989 requested a full tablet report, sending...
I20260808 23:00:08.368361 566 ts_manager.cc:194] Registered new tserver with Master: fb5e5477f5cb4c228b9a72e3aea99d59 (127.23.194.1:32781)
I20260808 23:00:08.369239 566 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:60621
WARNING: no policy specified for kudu/127.23.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
I20260808 23:00:08.388171 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:32989
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:41579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:08.467751 886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:08.467933 886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:08.467954 886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:08.469028 886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:08.469061 886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:08.469085 886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:00:08.470553 886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:41579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:32989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:08.470731 886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:08.470904 886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:08.475579 905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:08.475675 895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:08.475589 899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:08.475770 886 server_base.cc:1034] running on GCE node
I20260808 23:00:08.475970 886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:08.476207 886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:08.477336 886 hybrid_clock.cc:648] HybridClock initialized: now 1786230008477323 us; error 32 us; skew 500 ppm
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230008, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:08.479687 886 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 23:00:08.480093 886 webserver.cc:460] Webserver started at http://127.23.194.2:38201/ using document root <none> and password file <none>
I20260808 23:00:08.480273 886 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:08.480312 886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:08.480400 886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:08.481343 886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "73bc1814627e4945b86bf5397a4fcb77"
format_stamp: "Formatted at 2026-08-08 23:00:08 on dist-test-slave-f0tn"
server_key: "7ad2b4f31159ca9b78e9223d715b1beb"
server_key_iv: "d161fbb7e76ddc71a56eaaed1a4471a7"
server_key_version: "encryptionkey@0"
I20260808 23:00:08.481693 886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "73bc1814627e4945b86bf5397a4fcb77"
format_stamp: "Formatted at 2026-08-08 23:00:08 on dist-test-slave-f0tn"
server_key: "7ad2b4f31159ca9b78e9223d715b1beb"
server_key_iv: "d161fbb7e76ddc71a56eaaed1a4471a7"
server_key_version: "encryptionkey@0"
I20260808 23:00:08.482877 886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:00:08.483508 917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:08.483652 886 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:08.483732 886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "73bc1814627e4945b86bf5397a4fcb77"
format_stamp: "Formatted at 2026-08-08 23:00:08 on dist-test-slave-f0tn"
server_key: "7ad2b4f31159ca9b78e9223d715b1beb"
server_key_iv: "d161fbb7e76ddc71a56eaaed1a4471a7"
server_key_version: "encryptionkey@0"
I20260808 23:00:08.483795 886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:08.495417 886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:08.497139 886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:08.497268 886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:08.497539 886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:08.497828 886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:08.497861 886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:08.497880 886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:08.497902 886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:08.503441 886 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:38421
I20260808 23:00:08.503486 1099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:38421 every 8 connection(s)
I20260808 23:00:08.503770 886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230008, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:08.511775 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:08.504403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:49461 (local address 127.23.194.62:32989)
0808 23:00:08.504479 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:08.504482 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:08.504640 (+ 158us) server_negotiation.cc:407] Connection header received
0808 23:00:08.505534 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:08.505535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:08.505545 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:08.505592 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:08.506121 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:08.506601 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:08.507478 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:08.507652 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:08.507697 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:08.509564 (+ 1867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:08.509566 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:08.509566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:08.509573 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:08.510620 (+ 1047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:08.510943 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:08.510943 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:08.510943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:08.510985 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:08.511158 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:08.511158 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:08.511158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:08.511247 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:08.511301 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:08.511585 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 23:00:08.511678 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 23:00:08.511920 1104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:08.504381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:32989 (local address 127.23.194.2:49461)
0808 23:00:08.504527 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:08.504561 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:08.505443 (+ 882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:08.505645 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:08.505648 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:08.505887 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:08.506058 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:08.506062 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:08.506666 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:08.506667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:08.507414 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:08.507416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:08.507706 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:08.507707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:08.507798 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:08.508170 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:08.508179 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:08.509474 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:08.510685 (+ 1211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:08.510690 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:08.510692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:08.510889 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:08.511028 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:08.511028 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:08.511029 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:08.511111 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:08.511299 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:08.511302 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:08.511445 (+ 143us) client_negotiation.cc:769] Sending connection context
0808 23:00:08.511547 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 23:00:08.511710 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260808 23:00:08.512416 1101 heartbeater.cc:347] Connected to a master server at 127.23.194.62:32989
I20260808 23:00:08.512523 1101 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:08.512712 1101 heartbeater.cc:511] Master 127.23.194.62:32989 requested a full tablet report, sending...
I20260808 23:00:08.512849 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 886
I20260808 23:00:08.512959 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:00:08.513129 566 ts_manager.cc:194] Registered new tserver with Master: 73bc1814627e4945b86bf5397a4fcb77 (127.23.194.2:38421)
I20260808 23:00:08.513170 24328 external_mini_cluster.cc:1428] Setting key 50f89ed93b73e0b152c308175b7131c1
I20260808 23:00:08.513617 566 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:49461
WARNING: no policy specified for kudu/127.23.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
I20260808 23:00:08.537472 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:32989
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:41579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:08.617236 1112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:08.617443 1112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:08.617466 1112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:08.618531 1112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:08.618563 1112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:08.618587 1112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:00:08.620093 1112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:41579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:32989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:08.620289 1112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:08.620465 1112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:08.625172 1122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:08.625205 1121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:08.625170 1124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:08.625509 1112 server_base.cc:1034] running on GCE node
I20260808 23:00:08.625733 1112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:08.625965 1112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:08.627104 1112 hybrid_clock.cc:648] HybridClock initialized: now 1786230008627090 us; error 30 us; skew 500 ppm
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230008, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:08.629547 1112 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 23:00:08.629961 1112 webserver.cc:460] Webserver started at http://127.23.194.3:44835/ using document root <none> and password file <none>
I20260808 23:00:08.630195 1112 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:08.630244 1112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:08.630371 1112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:08.631393 1112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "137fd0fa322f4066a4528c2cc91e37b5"
format_stamp: "Formatted at 2026-08-08 23:00:08 on dist-test-slave-f0tn"
server_key: "dcfc76f63e35dc9695226a651cf14fab"
server_key_iv: "c13fd902339b7d2719609cc66f01131c"
server_key_version: "encryptionkey@0"
I20260808 23:00:08.631742 1112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "137fd0fa322f4066a4528c2cc91e37b5"
format_stamp: "Formatted at 2026-08-08 23:00:08 on dist-test-slave-f0tn"
server_key: "dcfc76f63e35dc9695226a651cf14fab"
server_key_iv: "c13fd902339b7d2719609cc66f01131c"
server_key_version: "encryptionkey@0"
I20260808 23:00:08.633011 1112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:00:08.633762 1138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:08.633912 1112 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:08.633998 1112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "137fd0fa322f4066a4528c2cc91e37b5"
format_stamp: "Formatted at 2026-08-08 23:00:08 on dist-test-slave-f0tn"
server_key: "dcfc76f63e35dc9695226a651cf14fab"
server_key_iv: "c13fd902339b7d2719609cc66f01131c"
server_key_version: "encryptionkey@0"
I20260808 23:00:08.634061 1112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:08.651114 1112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:08.653465 1112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:08.653614 1112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:08.653863 1112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:08.654243 1112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:08.654285 1112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:08.654316 1112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:08.654340 1112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:08.659771 1112 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:45573
I20260808 23:00:08.659845 1332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:45573 every 8 connection(s)
I20260808 23:00:08.660132 1112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 23:00:08.662385 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 1112
I20260808 23:00:08.662492 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:00:08.662694 24328 external_mini_cluster.cc:1428] Setting key f6d65cdc141ff6bcbf08404f36db6581
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230008, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:08.669279 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:08.660850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:46001 (local address 127.23.194.62:32989)
0808 23:00:08.660939 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:08.660942 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:08.661150 (+ 208us) server_negotiation.cc:407] Connection header received
0808 23:00:08.662124 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:08.662125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:08.662136 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:08.662200 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:08.663013 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:08.663718 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:08.664830 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:08.665027 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:08.665081 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:08.667173 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:08.667174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:08.667174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:08.667182 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:08.668253 (+ 1071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:08.668589 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:08.668589 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:08.668589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:08.668634 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:08.668805 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:08.668805 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:08.668806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:08.668900 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:08.668942 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:08.669143 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 23:00:08.669212 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:00:08.669502 1337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:08.660820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:32989 (local address 127.23.194.3:46001)
0808 23:00:08.661017 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:08.661067 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:08.662035 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:08.662283 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:08.662287 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:08.662656 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:08.662922 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:08.662929 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:08.663805 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:08.663806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:08.664768 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:08.664771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:08.665087 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:08.665087 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:08.665183 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:08.665715 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:08.665724 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:08.667069 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:08.668318 (+ 1249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:08.668324 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:08.668325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:08.668533 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:08.668676 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:08.668677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:08.668677 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:08.668765 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:08.668939 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:08.668941 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:08.669053 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 23:00:08.669129 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 23:00:08.669262 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":103,"threads_started":1}
I20260808 23:00:08.669939 1335 heartbeater.cc:347] Connected to a master server at 127.23.194.62:32989
I20260808 23:00:08.670028 1335 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:08.670179 1335 heartbeater.cc:511] Master 127.23.194.62:32989 requested a full tablet report, sending...
I20260808 23:00:08.670496 565 ts_manager.cc:194] Registered new tserver with Master: 137fd0fa322f4066a4528c2cc91e37b5 (127.23.194.3:45573)
I20260808 23:00:08.670873 565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:46001
I20260808 23:00:08.674337 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/test-admin.keytab.
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:08.689543 24328 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 23:00:08.691047 1344 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:08.690393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54674 (local address 127.23.194.1:32781)
0808 23:00:08.690550 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:08.690554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:08.690564 (+ 10us) server_negotiation.cc:407] Connection header received
0808 23:00:08.690591 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:08.690593 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:08.690606 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:08.690671 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:08.690967 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54674: BlockingRecv error: recv got EOF from 127.0.0.1:54674 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
W20260808 23:00:08.691185 24328 init.cc:244] Error while opening credential cache '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/krb5cc)
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:08.692076 24328 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:08.692858 24328 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.1@KRBTEST.COM
I20260808 23:00:08.699976 1344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:08.693368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54678 (local address 127.23.194.1:32781)
0808 23:00:08.693445 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:08.693447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:08.693481 (+ 34us) server_negotiation.cc:407] Connection header received
0808 23:00:08.693535 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:08.693536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:08.693545 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:08.693584 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:08.693926 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:08.694686 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:08.695421 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:08.695811 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:08.695866 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:08.697917 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:08.697925 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:08.697926 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:08.697932 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:08.699153 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:08.699430 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:08.699433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:08.699433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:08.699481 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:08.699599 (+ 118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:08.699599 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:08.699599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:08.699710 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:08.699817 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:08.699850 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 23:00:08.699909 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260808 23:00:08.701484 1344 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:08.701060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54682 (local address 127.23.194.1:32781)
0808 23:00:08.701113 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:08.701116 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:08.701123 (+ 7us) server_negotiation.cc:407] Connection header received
0808 23:00:08.701176 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:08.701176 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:08.701185 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:08.701223 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:08.701442 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54682: BlockingRecv error: recv got EOF from 127.0.0.1:54682 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 23:00:08.701550 24328 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/krb5cc' of type FILE
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:08.702375 24328 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:08.703096 24328 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.1@KRBTEST.COM
I20260808 23:00:08.708885 1344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:08.703517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54698 (local address 127.23.194.1:32781)
0808 23:00:08.703554 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:08.703556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:08.703594 (+ 38us) server_negotiation.cc:407] Connection header received
0808 23:00:08.703648 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:08.703649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:08.703659 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:08.703702 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:08.704028 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:08.704474 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:08.705112 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:08.705305 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:08.705357 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:08.707123 (+ 1766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:08.707124 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:08.707125 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:08.707130 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:08.708122 (+ 992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:08.708391 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:08.708391 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:08.708392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:08.708432 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:08.708546 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:08.708547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:08.708547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:08.708619 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:08.708676 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:08.708756 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 23:00:08.708810 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 23:00:08.710263 1344 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:08.709745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54712 (local address 127.23.194.1:32781)
0808 23:00:08.709778 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:08.709779 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:08.709807 (+ 28us) server_negotiation.cc:407] Connection header received
0808 23:00:08.709850 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:08.709850 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:08.709857 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:08.709895 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:08.710222 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54712: BlockingRecv error: recv got EOF from 127.0.0.1:54712 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260808 23:00:08.710357 24328 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786229981209642-24328-0/krb5kdc/krb5cc' of type FILE
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:08.711189 24328 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:08.711939 24328 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.1@KRBTEST.COM
I20260808 23:00:08.717911 1344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:08.712394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54718 (local address 127.23.194.1:32781)
0808 23:00:08.712431 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:08.712433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:08.712497 (+ 64us) server_negotiation.cc:407] Connection header received
0808 23:00:08.712552 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:08.712553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:08.712561 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:08.712602 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:08.712932 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:08.713567 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:08.714199 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:08.714390 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:08.714440 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:08.716151 (+ 1711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:08.716152 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:08.716152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:08.716163 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:08.717122 (+ 959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:08.717415 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:08.717416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:08.717416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:08.717468 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:08.717601 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:08.717601 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:08.717602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:08.717677 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:08.717735 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:08.717806 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 23:00:08.717857 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 23:00:08.718402 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 669
I20260808 23:00:08.731456 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 886
I20260808 23:00:08.754150 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 1112
I20260808 23:00:08.763479 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 509
2026-08-08T23:00:08Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1973 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[1363](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[1363](info): set up 2 sockets
Aug 08 23:00:08 dist-test-slave-f0tn krb5kdc[1363](info): commencing operation
krb5kdc: starting...
W20260808 23:00:09.890625 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:00:09 dist-test-slave-f0tn krb5kdc[1363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:00:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:00:09Z Disabled control of system clock
WARNING: no policy specified for oryx/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.62.keytab.
Entry for principal oryx/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.62.keytab.
I20260808 23:00:10.005591 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:35945
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.62.keytab
--principal=oryx/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:35945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:10.087256 1380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:10.087469 1380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:10.087491 1380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:10.088639 1380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:00:10.088683 1380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:10.088696 1380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:10.088708 1380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:00:10.088719 1380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:00:10.090217 1380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:35945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.62.keytab
--principal=oryx/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:35945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:10.090428 1380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:10.090617 1380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:10.095464 1386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:10.095533 1380 server_base.cc:1034] running on GCE node
W20260808 23:00:10.095476 1390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:10.095481 1387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:10.095929 1380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:10.096249 1380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:10.097438 1380 hybrid_clock.cc:648] HybridClock initialized: now 1786230010097421 us; error 34 us; skew 500 ppm
Aug 08 23:00:10 dist-test-slave-f0tn krb5kdc[1363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230010, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:10.106281 1380 init.cc:377] Logged in from keytab as oryx/127.23.194.62@KRBTEST.COM (short username oryx)
I20260808 23:00:10.106745 1380 webserver.cc:460] Webserver started at http://127.23.194.62:39589/ using document root <none> and password file <none>
I20260808 23:00:10.106971 1380 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:10.107016 1380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:10.107157 1380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:10.108198 1380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "bc466d1f87bb4f4387cffdf35a21a5e5"
format_stamp: "Formatted at 2026-08-08 23:00:10 on dist-test-slave-f0tn"
server_key: "2d6b51c195b360e63adb540ad8d5026d"
server_key_iv: "87b2442b6af3d0d9b794b35c1631739f"
server_key_version: "encryptionkey@0"
I20260808 23:00:10.108529 1380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "bc466d1f87bb4f4387cffdf35a21a5e5"
format_stamp: "Formatted at 2026-08-08 23:00:10 on dist-test-slave-f0tn"
server_key: "2d6b51c195b360e63adb540ad8d5026d"
server_key_iv: "87b2442b6af3d0d9b794b35c1631739f"
server_key_version: "encryptionkey@0"
I20260808 23:00:10.109827 1380 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 23:00:10.110517 1399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:10.110661 1380 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:00:10.110743 1380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "bc466d1f87bb4f4387cffdf35a21a5e5"
format_stamp: "Formatted at 2026-08-08 23:00:10 on dist-test-slave-f0tn"
server_key: "2d6b51c195b360e63adb540ad8d5026d"
server_key_iv: "87b2442b6af3d0d9b794b35c1631739f"
server_key_version: "encryptionkey@0"
I20260808 23:00:10.110806 1380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:10.125511 1380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:10.127410 1380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:10.127538 1380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:10.131511 1380 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:35945
I20260808 23:00:10.131564 1490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:35945 every 8 connection(s)
I20260808 23:00:10.131887 1380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:00:10.132861 1493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:10.135105 1493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5: Bootstrap starting.
I20260808 23:00:10.135671 1493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:10.135938 1493 log.cc:824] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:10.136627 1493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5: No bootstrap required, opened a new log
I20260808 23:00:10.138245 1493 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } }
I20260808 23:00:10.138384 1493 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:10.138406 1493 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc466d1f87bb4f4387cffdf35a21a5e5, State: Initialized, Role: FOLLOWER
I20260808 23:00:10.138553 1493 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } }
I20260808 23:00:10.138616 1493 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:00:10.138639 1493 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:00:10.138695 1493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:10.139482 1493 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } }
I20260808 23:00:10.139575 1493 leader_election.cc:302] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc466d1f87bb4f4387cffdf35a21a5e5; no voters:
I20260808 23:00:10.139739 1493 leader_election.cc:288] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:00:10.139776 1497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:10.139907 1497 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [term 1 LEADER]: Becoming Leader. State: Replica: bc466d1f87bb4f4387cffdf35a21a5e5, State: Running, Role: LEADER
I20260808 23:00:10.140003 1493 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:00:10.140003 1497 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } }
I20260808 23:00:10.140425 1500 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc466d1f87bb4f4387cffdf35a21a5e5. Latest consensus state: current_term: 1 leader_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } }
I20260808 23:00:10.140516 1500 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:10.140435 1498 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } }
I20260808 23:00:10.140619 1498 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:10.140616 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 1380
I20260808 23:00:10.140736 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:00:10.140774 1504 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:00:10.140937 24328 external_mini_cluster.cc:1428] Setting key 07417bebbf994acc10f17e20f2ff2847
I20260808 23:00:10.141288 1504 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:00:10.142818 1504 catalog_manager.cc:1347] Generated new cluster ID: ad6ba00deee14b4ea2be0cd0685da656
I20260808 23:00:10.142872 1504 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 23:00:10 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM
I20260808 23:00:10.150098 1504 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:00:10.150774 1504 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:00:10.151245 1513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.141476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56864 (local address 127.23.194.62:35945)
0808 23:00:10.141665 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:10.141675 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:10.142680 (+ 1005us) server_negotiation.cc:407] Connection header received
0808 23:00:10.142819 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:10.142825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:10.142872 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:10.143010 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:10.143605 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.144509 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.145244 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.145713 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.145820 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.147980 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:10.147985 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:10.147985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:10.147997 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:10.149562 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.149884 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.149887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.149887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.150005 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.150255 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.150256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.150257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.150537 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:10.150691 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:10.150803 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 23:00:10.151017 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":63,"threads_started":1}
I20260808 23:00:10.156256 1504 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5: Generated new TSK 0
I20260808 23:00:10.156420 1504 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.1.keytab.
Entry for principal oryx/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.1.keytab.
I20260808 23:00:10.190090 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35945
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.1.keytab
--principal=oryx/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:10.270582 1534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:10.270783 1534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:10.270805 1534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:10.271883 1534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:10.271914 1534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:10.271938 1534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:00:10.273355 1534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.1.keytab
--principal=oryx/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:10.273571 1534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:10.273753 1534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:10.278723 1545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:10.278710 1540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:10.278707 1543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:10.278759 1534 server_base.cc:1034] running on GCE node
I20260808 23:00:10.279042 1534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:10.279285 1534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:10.280424 1534 hybrid_clock.cc:648] HybridClock initialized: now 1786230010280407 us; error 30 us; skew 500 ppm
Aug 08 23:00:10 dist-test-slave-f0tn krb5kdc[1363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230010, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:10.282886 1534 init.cc:377] Logged in from keytab as oryx/127.23.194.1@KRBTEST.COM (short username oryx)
I20260808 23:00:10.283318 1534 webserver.cc:460] Webserver started at http://127.23.194.1:40025/ using document root <none> and password file <none>
I20260808 23:00:10.283533 1534 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:10.283583 1534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:10.283715 1534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:10.284751 1534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "330d9fc4eb7b4a089b33ab13a463351c"
format_stamp: "Formatted at 2026-08-08 23:00:10 on dist-test-slave-f0tn"
server_key: "a8d9852da0a63a9c2ecfd6ac0fcb1ec2"
server_key_iv: "feb94d2d64bf4fd125ce5acebe726081"
server_key_version: "encryptionkey@0"
I20260808 23:00:10.285081 1534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "330d9fc4eb7b4a089b33ab13a463351c"
format_stamp: "Formatted at 2026-08-08 23:00:10 on dist-test-slave-f0tn"
server_key: "a8d9852da0a63a9c2ecfd6ac0fcb1ec2"
server_key_iv: "feb94d2d64bf4fd125ce5acebe726081"
server_key_version: "encryptionkey@0"
I20260808 23:00:10.286408 1534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 23:00:10.287117 1553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:10.287269 1534 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:00:10.287351 1534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "330d9fc4eb7b4a089b33ab13a463351c"
format_stamp: "Formatted at 2026-08-08 23:00:10 on dist-test-slave-f0tn"
server_key: "a8d9852da0a63a9c2ecfd6ac0fcb1ec2"
server_key_iv: "feb94d2d64bf4fd125ce5acebe726081"
server_key_version: "encryptionkey@0"
I20260808 23:00:10.287418 1534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:10.305075 1534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:10.307245 1534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:10.307389 1534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:10.307621 1534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:10.307922 1534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:10.307955 1534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:10.307983 1534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:10.308008 1534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:10.313359 1534 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:36397
I20260808 23:00:10.313447 1730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:36397 every 8 connection(s)
I20260808 23:00:10.313752 1534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:00:10.315001 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 1534
I20260808 23:00:10.315094 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:00:10.315311 24328 external_mini_cluster.cc:1428] Setting key 82f3af078a8c10b604e5fc8625e134e8
Aug 08 23:00:10 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230010, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM
I20260808 23:00:10.323002 1513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.314397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:33157 (local address 127.23.194.62:35945)
0808 23:00:10.314458 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:10.314461 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:10.314696 (+ 235us) server_negotiation.cc:407] Connection header received
0808 23:00:10.315211 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:10.315213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:10.315226 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:10.315298 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:10.316308 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.316958 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.318198 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.318397 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.318449 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.320445 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:10.320446 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:10.320446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:10.320454 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:10.321780 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.322128 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.322129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.322130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.322185 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.322396 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.322396 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.322397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.322504 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:10.322561 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:10.322815 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 23:00:10.322909 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 23:00:10.323200 1733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.314378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.23.194.1:33157)
0808 23:00:10.314574 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:10.314612 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:10.315097 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:10.315356 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:10.315359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:10.316016 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:10.316217 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:10.316222 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.317036 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.317037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:10.318102 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:10.318106 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.318469 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.318469 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:10.318611 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.319073 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:10.319082 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:10.320361 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:10.321850 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:10.321857 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:10.321859 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:10.322064 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:10.322238 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:10.322239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:10.322239 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:10.322341 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:10.322558 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:10.322564 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:10.322698 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 23:00:10.322784 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 23:00:10.322931 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
I20260808 23:00:10.323671 1731 heartbeater.cc:347] Connected to a master server at 127.23.194.62:35945
WARNING: no policy specified for oryx/127.23.194.2@KRBTEST.COM; defaulting to no policy
I20260808 23:00:10.323827 1731 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:10.324033 1731 heartbeater.cc:511] Master 127.23.194.62:35945 requested a full tablet report, sending...
I20260808 23:00:10.324572 1431 ts_manager.cc:194] Registered new tserver with Master: 330d9fc4eb7b4a089b33ab13a463351c (127.23.194.1:36397)
I20260808 23:00:10.325237 1431 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:33157
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.2.keytab.
Entry for principal oryx/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.2.keytab.
I20260808 23:00:10.340533 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35945
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.2.keytab
--principal=oryx/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:10.423848 1739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:10.424111 1739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:10.424139 1739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:10.425320 1739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:10.425408 1739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:10.425462 1739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:00:10.427029 1739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.2.keytab
--principal=oryx/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:10.427270 1739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:10.427507 1739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:10.432466 1746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:10.432459 1749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:10.432669 1739 server_base.cc:1034] running on GCE node
W20260808 23:00:10.432457 1747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:10.432994 1739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:10.433207 1739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:10.434345 1739 hybrid_clock.cc:648] HybridClock initialized: now 1786230010434328 us; error 34 us; skew 500 ppm
Aug 08 23:00:10 dist-test-slave-f0tn krb5kdc[1363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230010, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:10.436721 1739 init.cc:377] Logged in from keytab as oryx/127.23.194.2@KRBTEST.COM (short username oryx)
I20260808 23:00:10.437130 1739 webserver.cc:460] Webserver started at http://127.23.194.2:33649/ using document root <none> and password file <none>
I20260808 23:00:10.437325 1739 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:10.437371 1739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:10.437534 1739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:10.438565 1739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "c3b85349630b484dbd4ea782a8009075"
format_stamp: "Formatted at 2026-08-08 23:00:10 on dist-test-slave-f0tn"
server_key: "5e926782bbf85a1a61c3995d862ee384"
server_key_iv: "33d19dcd14621ed252310769200c5bae"
server_key_version: "encryptionkey@0"
I20260808 23:00:10.438897 1739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "c3b85349630b484dbd4ea782a8009075"
format_stamp: "Formatted at 2026-08-08 23:00:10 on dist-test-slave-f0tn"
server_key: "5e926782bbf85a1a61c3995d862ee384"
server_key_iv: "33d19dcd14621ed252310769200c5bae"
server_key_version: "encryptionkey@0"
I20260808 23:00:10.440237 1739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:00:10.440951 1762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:10.441098 1739 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:10.441210 1739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "c3b85349630b484dbd4ea782a8009075"
format_stamp: "Formatted at 2026-08-08 23:00:10 on dist-test-slave-f0tn"
server_key: "5e926782bbf85a1a61c3995d862ee384"
server_key_iv: "33d19dcd14621ed252310769200c5bae"
server_key_version: "encryptionkey@0"
I20260808 23:00:10.441270 1739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:10.454268 1739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:10.455920 1739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:10.456043 1739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:10.456238 1739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:10.456530 1739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:10.456564 1739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:10.456591 1739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:10.456615 1739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:10.462157 1739 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:42459
I20260808 23:00:10.462196 1929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:42459 every 8 connection(s)
I20260808 23:00:10.462483 1739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:00:10.465930 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 1739
I20260808 23:00:10.466022 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:00:10.466192 24328 external_mini_cluster.cc:1428] Setting key 74b84da891d270304be9b377ac04c9ae
Aug 08 23:00:10 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230010, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM
I20260808 23:00:10.471520 1513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.463167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:55577 (local address 127.23.194.62:35945)
0808 23:00:10.463229 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:10.463231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:10.463389 (+ 158us) server_negotiation.cc:407] Connection header received
0808 23:00:10.463844 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:10.463845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:10.463854 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:10.463905 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:10.464752 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.465228 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.466202 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.466486 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.466548 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.468915 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:10.468916 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:10.468916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:10.468923 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:10.470222 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.470641 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.470642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.470642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.470700 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.470956 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.470957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.470957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.471050 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:10.471093 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:10.471394 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 23:00:10.471456 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 23:00:10.471774 1932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.463122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.23.194.2:55577)
0808 23:00:10.463273 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:10.463312 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:10.463769 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:10.463956 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:10.463959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:10.464498 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:10.464685 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:10.464689 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.465290 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.465291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:10.466117 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:10.466119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.466578 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.466579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:10.466693 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.467194 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:10.467203 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:10.468825 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:10.470301 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:10.470308 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:10.470310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:10.470576 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:10.470749 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:10.470750 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:10.470750 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:10.470893 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:10.471103 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:10.471110 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:10.471269 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 23:00:10.471380 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 23:00:10.471538 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
I20260808 23:00:10.472229 1930 heartbeater.cc:347] Connected to a master server at 127.23.194.62:35945
I20260808 23:00:10.472337 1930 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:10.472529 1930 heartbeater.cc:511] Master 127.23.194.62:35945 requested a full tablet report, sending...
I20260808 23:00:10.472893 1431 ts_manager.cc:194] Registered new tserver with Master: c3b85349630b484dbd4ea782a8009075 (127.23.194.2:42459)
I20260808 23:00:10.473255 1431 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:55577
WARNING: no policy specified for oryx/127.23.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.3.keytab.
Entry for principal oryx/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.3.keytab.
I20260808 23:00:10.489352 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35945
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.3.keytab
--principal=oryx/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:10.570772 1935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:10.570974 1935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:10.570995 1935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:10.572095 1935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:10.572134 1935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:10.572157 1935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:00:10.573611 1935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.3.keytab
--principal=oryx/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:35945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:10.573815 1935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:10.574002 1935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:10.578869 1941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:10.578907 1944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:10.578949 1935 server_base.cc:1034] running on GCE node
W20260808 23:00:10.578907 1942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:10.579265 1935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:10.579502 1935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:10.580641 1935 hybrid_clock.cc:648] HybridClock initialized: now 1786230010580625 us; error 32 us; skew 500 ppm
Aug 08 23:00:10 dist-test-slave-f0tn krb5kdc[1363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230010, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:10.583042 1935 init.cc:377] Logged in from keytab as oryx/127.23.194.3@KRBTEST.COM (short username oryx)
I20260808 23:00:10.583468 1935 webserver.cc:460] Webserver started at http://127.23.194.3:37483/ using document root <none> and password file <none>
I20260808 23:00:10.583671 1935 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:10.583710 1935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:10.583801 1935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:10.584758 1935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4"
format_stamp: "Formatted at 2026-08-08 23:00:10 on dist-test-slave-f0tn"
server_key: "80ff0c419d5e11468ed5c28cdcc49526"
server_key_iv: "ec6a40990bc96d248795e16fed042793"
server_key_version: "encryptionkey@0"
I20260808 23:00:10.585034 1935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4"
format_stamp: "Formatted at 2026-08-08 23:00:10 on dist-test-slave-f0tn"
server_key: "80ff0c419d5e11468ed5c28cdcc49526"
server_key_iv: "ec6a40990bc96d248795e16fed042793"
server_key_version: "encryptionkey@0"
I20260808 23:00:10.586257 1935 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 23:00:10.586884 1955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:10.587041 1935 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:00:10.587122 1935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4"
format_stamp: "Formatted at 2026-08-08 23:00:10 on dist-test-slave-f0tn"
server_key: "80ff0c419d5e11468ed5c28cdcc49526"
server_key_iv: "ec6a40990bc96d248795e16fed042793"
server_key_version: "encryptionkey@0"
I20260808 23:00:10.587200 1935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:10.602711 1935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:10.604444 1935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:10.604550 1935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:10.604732 1935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:10.604975 1935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:10.605005 1935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:10.605024 1935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:10.605041 1935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:10.610471 1935 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:40999
I20260808 23:00:10.610522 2124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:40999 every 8 connection(s)
I20260808 23:00:10.610829 1935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 23:00:10.613903 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 1935
I20260808 23:00:10.613996 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:00:10.614194 24328 external_mini_cluster.cc:1428] Setting key aad5266bb7743b6ca4ffe8a6f6eebf0c
Aug 08 23:00:10 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230010, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM
I20260808 23:00:10.619375 1513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.611498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:58743 (local address 127.23.194.62:35945)
0808 23:00:10.611577 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:10.611579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:10.611744 (+ 165us) server_negotiation.cc:407] Connection header received
0808 23:00:10.612199 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:10.612200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:10.612210 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:10.612253 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:10.613143 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.613725 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.614927 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.615115 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.615169 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.617068 (+ 1899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:10.617069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:10.617070 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:10.617077 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:10.618174 (+ 1097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.618523 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.618524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.618524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.618578 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.618790 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.618791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.618791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.618898 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:10.618955 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:10.619207 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 23:00:10.619286 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 23:00:10.619535 2127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.611466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.23.194.3:58743)
0808 23:00:10.611620 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:10.611668 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:10.612122 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:10.612309 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:10.612312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:10.612871 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:10.613075 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:10.613079 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.613807 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.613808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:10.614854 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:10.614857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.615197 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.615198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:10.615330 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.615729 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:10.615736 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:10.616985 (+ 1249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:10.618244 (+ 1259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:10.618250 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:10.618252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:10.618457 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:10.618631 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:10.618632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:10.618632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:10.618733 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:10.618954 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:10.618961 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:10.619090 (+ 129us) client_negotiation.cc:769] Sending connection context
0808 23:00:10.619184 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 23:00:10.619332 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
I20260808 23:00:10.619927 2125 heartbeater.cc:347] Connected to a master server at 127.23.194.62:35945
I20260808 23:00:10.620011 2125 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:10.620191 2125 heartbeater.cc:511] Master 127.23.194.62:35945 requested a full tablet report, sending...
I20260808 23:00:10.620522 1431 ts_manager.cc:194] Registered new tserver with Master: 5d4dcce6f4d54254a9b9a8c62290a0e4 (127.23.194.3:40999)
I20260808 23:00:10.620872 1431 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:58743
I20260808 23:00:10.625813 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.23.194.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.194.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.194.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.61.keytab.
Entry for principal oryx/127.23.194.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.61.keytab.
I20260808 23:00:10.648979 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.61:38237
--webserver_interface=127.23.194.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.61.keytab
--principal=oryx/127.23.194.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:35945,127.23.194.61:38237
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:10.730481 2133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:10.730695 2133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:10.730732 2133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:10.731848 2133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:00:10.731890 2133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:10.731910 2133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:10.731927 2133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:00:10.731942 2133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:00:10.733417 2133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:35945,127.23.194.61:38237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.61.keytab
--principal=oryx/127.23.194.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.61:38237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.194.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
W20260808 23:00:10.733537 2133 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.23.194.62:35945,127.23.194.61:38237'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 23:00:10.733680 2133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:10.733875 2133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 23:00:10.738621 2133 server_base.cc:1034] running on GCE node
W20260808 23:00:10.738626 2143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:10.738722 2141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:10.738854 2140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:10.739077 2133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:10.739363 2133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:10.740512 2133 hybrid_clock.cc:648] HybridClock initialized: now 1786230010740490 us; error 34 us; skew 500 ppm
Aug 08 23:00:10 dist-test-slave-f0tn krb5kdc[1363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230010, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:10.743245 2133 init.cc:377] Logged in from keytab as oryx/127.23.194.61@KRBTEST.COM (short username oryx)
I20260808 23:00:10.743698 2133 webserver.cc:460] Webserver started at http://127.23.194.61:35203/ using document root <none> and password file <none>
I20260808 23:00:10.743903 2133 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:10.743952 2133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:10.744108 2133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:10.745184 2133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/data/instance:
uuid: "faec855a53e74875a88bbb0c1a5107a6"
format_stamp: "Formatted at 2026-08-08 23:00:10 on dist-test-slave-f0tn"
server_key: "a6ea22897651aed6b34392fb90118a5b"
server_key_iv: "9d44637108e6b6bbbcd2ce245eb71c24"
server_key_version: "encryptionkey@0"
I20260808 23:00:10.745553 2133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/wal/instance:
uuid: "faec855a53e74875a88bbb0c1a5107a6"
format_stamp: "Formatted at 2026-08-08 23:00:10 on dist-test-slave-f0tn"
server_key: "a6ea22897651aed6b34392fb90118a5b"
server_key_iv: "9d44637108e6b6bbbcd2ce245eb71c24"
server_key_version: "encryptionkey@0"
I20260808 23:00:10.746815 2133 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 23:00:10.747494 2152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:10.747684 2133 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:00:10.747769 2133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/wal
uuid: "faec855a53e74875a88bbb0c1a5107a6"
format_stamp: "Formatted at 2026-08-08 23:00:10 on dist-test-slave-f0tn"
server_key: "a6ea22897651aed6b34392fb90118a5b"
server_key_iv: "9d44637108e6b6bbbcd2ce245eb71c24"
server_key_version: "encryptionkey@0"
I20260808 23:00:10.747835 2133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:10.761229 2133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:10.763185 2133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:10.763319 2133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:10.767132 2133 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.61:38237
I20260808 23:00:10.767171 2238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.61:38237 every 8 connection(s)
I20260808 23:00:10.767488 2133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/data/info.pb
I20260808 23:00:10.768435 2241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:10.769938 2241 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:00:10.774184 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 2133
Aug 08 23:00:10 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230010, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.61@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM
I20260808 23:00:10.778230 1513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.770575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56870 (local address 127.23.194.62:35945)
0808 23:00:10.770617 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:10.770619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:10.770812 (+ 193us) server_negotiation.cc:407] Connection header received
0808 23:00:10.771232 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:10.771233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:10.771242 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:10.771295 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:10.771827 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.772303 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.773190 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.773357 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.773432 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.775563 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:10.775565 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:10.775565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:10.775571 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:10.776830 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.777184 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.777185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.777186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.777257 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.777534 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.777535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.777535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.777751 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:10.777816 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:10.778100 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 23:00:10.778168 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 23:00:10.778457 2244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.770542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.0.0.1:56870)
0808 23:00:10.770706 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:10.770745 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:10.771161 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:10.771353 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:10.771356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:10.771573 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:10.771768 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:10.771772 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.772359 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.772360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:10.773130 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:10.773133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.773429 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.773429 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:10.773520 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.773904 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:10.773911 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:10.775473 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:10.776911 (+ 1438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:10.776918 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:10.776920 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:10.777119 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:10.777318 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:10.777319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:10.777319 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:10.777472 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:10.777815 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:10.777821 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:10.777970 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 23:00:10.778086 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 23:00:10.778234 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
I20260808 23:00:10.779176 2241 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:00:10 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.194.61@KRBTEST.COM
I20260808 23:00:10.782372 2247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.774592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36002 (local address 127.23.194.61:38237)
0808 23:00:10.774723 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:10.774727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:10.774742 (+ 15us) server_negotiation.cc:407] Connection header received
0808 23:00:10.774763 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:10.774765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:10.774779 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:10.774836 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:10.775503 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.776152 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.776881 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.777209 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.777278 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.779741 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:10.779749 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:10.779749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:10.779755 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:10.781181 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.781563 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.781566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.781566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.781627 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.781820 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.781821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.781821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.781945 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:10.782087 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:10.782181 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 23:00:10.782276 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
Aug 08 23:00:10 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230010, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.61@KRBTEST.COM for oryx/127.23.194.61@KRBTEST.COM
I20260808 23:00:10.785775 2244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.779391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:38237 (local address 127.0.0.1:36008)
0808 23:00:10.779581 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:10.779590 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:10.779641 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:10.779787 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:10.779787 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:10.779933 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:10.779987 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:10.779989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.780827 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.780828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:10.781302 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:10.781304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.781743 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.781744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:10.781813 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.782174 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:10.782178 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:10.783383 (+ 1205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:10.785059 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:10.785060 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:10.785060 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:10.785206 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:10.785339 (+ 133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:10.785340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:10.785340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:10.785407 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:10.785580 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:10.785580 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:10.785610 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 23:00:10.785656 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 23:00:10.785720 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177}
I20260808 23:00:10.785885 2248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.779443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36008 (local address 127.23.194.61:38237)
0808 23:00:10.779600 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:10.779601 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:10.779627 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:00:10.779702 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:10.779703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:10.779712 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:10.779786 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:10.780054 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.780751 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.781372 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.781667 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.781722 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.783451 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:10.783452 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:10.783453 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:10.783459 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:10.784990 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.785255 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.785256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.785256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.785299 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.785452 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.785452 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.785453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.785537 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:10.785584 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:10.785679 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 23:00:10.785763 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":138,"thread_start_us":107,"threads_started":1}
I20260808 23:00:10.786978 2241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6: Bootstrap starting.
I20260808 23:00:10.787555 2241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:10.787779 2241 log.cc:824] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:10.788393 2241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6: No bootstrap required, opened a new log
I20260808 23:00:10.789839 2241 raft_consensus.cc:348] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } }
I20260808 23:00:10.789949 2241 raft_consensus.cc:374] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:10.789970 2241 raft_consensus.cc:729] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faec855a53e74875a88bbb0c1a5107a6, State: Initialized, Role: FOLLOWER
I20260808 23:00:10.790060 2241 consensus_queue.cc:260] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } }
I20260808 23:00:10.790313 2241 sys_catalog.cc:564] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:00:10.790278 2252 sys_catalog.cc:455] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } } }
I20260808 23:00:10.790354 2252 sys_catalog.cc:458] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 23:00:10.791822 2268 catalog_manager.cc:1558] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:00:10.791882 2268 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:00:10.793432 2133 master_runner.cc:428] Detected that this master faec855a53e74875a88bbb0c1a5107a6 is joining an existing cluster
I20260808 23:00:10.793505 2133 master_runner.cc:432] Initiating AddMaster RPC to add 127.23.194.61:38237
I20260808 23:00:10.800536 2248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.794592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36022 (local address 127.23.194.61:38237)
0808 23:00:10.794625 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:10.794627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:10.794747 (+ 120us) server_negotiation.cc:407] Connection header received
0808 23:00:10.794827 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:10.794828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:10.794838 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:10.794886 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:10.795187 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.795625 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.796517 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.796829 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.796901 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.797832 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:10.797833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:10.797833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:10.797839 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:10.799394 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.799821 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.799822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.799822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.799874 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.800086 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.800087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.800087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.800199 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:10.800258 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:10.800375 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 23:00:10.800451 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 23:00:10.800736 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.794422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:38237 (local address 127.0.0.1:36022)
0808 23:00:10.794709 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:10.794715 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:10.794762 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:10.794890 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:10.794890 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:10.795083 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:10.795130 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:10.795131 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.795687 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.795688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:10.796436 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:10.796438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.796915 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.796916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:10.796971 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.797348 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:10.797353 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:10.797760 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:10.799478 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:10.799479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:10.799480 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:10.799749 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:10.799930 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:10.799931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:10.799931 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:10.800000 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:10.800255 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:10.800256 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:10.800292 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 23:00:10.800351 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 23:00:10.800504 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":67,"thread_start_us":75,"threads_started":1}
I20260808 23:00:10.800828 1513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.794461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56878 (local address 127.23.194.62:35945)
0808 23:00:10.794504 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:10.794506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:10.794651 (+ 145us) server_negotiation.cc:407] Connection header received
0808 23:00:10.794684 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:10.794685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:10.794696 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:10.794758 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:10.795134 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.795874 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.796737 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.796965 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.797029 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.798060 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:10.798062 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:10.798062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:10.798069 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:10.799492 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.799935 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.799936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.799936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.799981 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.800174 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.800175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.800175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.800308 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:10.800369 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:10.800546 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 23:00:10.800736 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 23:00:10.801087 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.794405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.0.0.1:56878)
0808 23:00:10.794576 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:10.794585 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:10.794643 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:10.794769 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:10.794769 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:10.794975 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:10.795059 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:10.795060 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.795951 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.795951 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:10.796671 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:10.796673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.797034 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.797035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:10.797096 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.797520 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:10.797524 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:10.797976 (+ 452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:10.799551 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:10.799552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:10.799552 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:10.799763 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:10.800034 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:10.800035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:10.800035 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:10.800101 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:10.800395 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:10.800396 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:10.800436 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 23:00:10.800893 (+ 457us) client_negotiation.cc:240] Negotiation successful
0808 23:00:10.800988 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
Aug 08 23:00:10 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230010, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.62@KRBTEST.COM for oryx/127.23.194.61@KRBTEST.COM
I20260808 23:00:10.809844 2248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.803412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36028 (local address 127.23.194.61:38237)
0808 23:00:10.803443 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:10.803445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:10.803604 (+ 159us) server_negotiation.cc:407] Connection header received
0808 23:00:10.803689 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:10.803689 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:10.803699 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:10.803746 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:10.804150 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.804595 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.805314 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.805548 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.805603 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.807613 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:10.807614 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:10.807614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:10.807621 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:10.808757 (+ 1136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.809099 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.809100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.809100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.809154 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.809314 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.809314 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.809314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.809452 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:10.809513 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:10.809684 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 23:00:10.809756 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 23:00:10.809852 2284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.803377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:38237 (local address 127.0.0.1:36028)
0808 23:00:10.803507 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:10.803541 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:10.803614 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:10.803760 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:10.803764 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:10.804031 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:10.804079 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:10.804081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.804664 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.804665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:10.805241 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:10.805242 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.805609 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.805610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:10.805694 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.806110 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:10.806114 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:10.807531 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:10.808828 (+ 1297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:10.808835 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:10.808837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:10.809033 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:10.809207 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:10.809207 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:10.809207 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:10.809265 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:10.809512 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:10.809520 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:10.809570 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 23:00:10.809641 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 23:00:10.809760 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":72,"threads_started":1}
I20260808 23:00:10.810431 1431 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.23.194.61:38237: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6"
member_type: NON_VOTER
last_known_addr {
host: "127.23.194.61"
port: 38237
}
attrs {
promote: true
}
}
dest_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5"
cas_config_opid_index: -1
I20260808 23:00:10.810755 1431 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: NON_VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: true } }
I20260808 23:00:10.811231 2286 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER faec855a53e74875a88bbb0c1a5107a6 (127.23.194.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: NON_VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: true } } }
I20260808 23:00:10.811522 2191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:10.812237 2191 raft_consensus.cc:1261] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 1 FOLLOWER]: Refusing update from remote peer bc466d1f87bb4f4387cffdf35a21a5e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 23:00:10.812289 2286 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.23.194.61:38237
I20260808 23:00:10.812614 2287 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: NON_VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:10.812759 2133 master.cc:537] Master@127.23.194.61:38237 shutting down...
W20260808 23:00:10.813398 1401 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260808 23:00:10.813678 2285 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: NON_VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: true } } }
I20260808 23:00:10.813758 2285 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: This master's current role is: LEADER
W20260808 23:00:10.814383 1401 consensus_peers.cc:487] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 -> Peer faec855a53e74875a88bbb0c1a5107a6 (127.23.194.61:38237): Couldn't send request to peer faec855a53e74875a88bbb0c1a5107a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 23:00:10.815997 2133 raft_consensus.cc:2229] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:00:10.816089 2133 raft_consensus.cc:2258] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:10.816139 2133 tablet_replica.cc:330] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6: stopping tablet replica
I20260808 23:00:10.829077 2133 master.cc:552] Master@127.23.194.61:38237 shutdown complete.
I20260808 23:00:10.829123 2133 master_runner.cc:305] Clearing existing system tablet
I20260808 23:00:10.829267 2133 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:10.829309 2133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:10.829962 2133 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:10.830380 2293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:10.830525 2133 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:10.830578 2133 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:10.830601 2133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/wal
uuid: "faec855a53e74875a88bbb0c1a5107a6"
format_stamp: "Formatted at 2026-08-08 23:00:10 on dist-test-slave-f0tn"
server_key: "a6ea22897651aed6b34392fb90118a5b"
server_key_iv: "9d44637108e6b6bbbcd2ce245eb71c24"
server_key_version: "encryptionkey@0"
I20260808 23:00:10.831290 2133 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:10.832628 2133 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 23:00:10.832684 2133 log.cc:1196] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 23:00:10.832849 2133 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6: Deleting consensus metadata
I20260808 23:00:10.832955 2133 master_runner.cc:315] Copying system tablet from 127.23.194.62:35945
I20260808 23:00:10.833556 2133 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6: tablet copy: Beginning tablet copy session from remote peer at address 127.23.194.62:35945
I20260808 23:00:10.839510 2305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.833871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.0.0.1:56884)
0808 23:00:10.833967 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:10.833977 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:10.834037 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:10.834171 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:10.834172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:10.834314 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:10.834366 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:10.834367 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.835259 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.835260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:10.835783 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:10.835784 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.836213 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.836213 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:10.836255 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.836699 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:10.836703 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:10.836966 (+ 263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:10.838658 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:10.838660 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:10.838660 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:10.838819 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:10.838999 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:10.839000 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:10.839000 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:10.839044 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:10.839275 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:10.839275 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:10.839308 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 23:00:10.839351 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 23:00:10.839426 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":62,"threads_started":1}
I20260808 23:00:10.839670 1513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.833937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56884 (local address 127.23.194.62:35945)
0808 23:00:10.833983 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:10.833985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:10.834023 (+ 38us) server_negotiation.cc:407] Connection header received
0808 23:00:10.834082 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:10.834083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:10.834093 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:10.834167 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:10.834424 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.835188 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.835850 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.836151 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.836219 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.837046 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:10.837048 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:10.837048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:10.837056 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:10.838582 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.838882 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.838883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.838884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.838946 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.839080 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.839081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.839081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.839212 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:10.839287 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:10.839378 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 23:00:10.839458 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 23:00:10.839990 1466 tablet_copy_service.cc:140] P bc466d1f87bb4f4387cffdf35a21a5e5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer faec855a53e74875a88bbb0c1a5107a6 ({username='oryx', principal='oryx/127.23.194.61@KRBTEST.COM'} at 127.0.0.1:56884)
I20260808 23:00:10.840163 1466 tablet_copy_service.cc:161] P bc466d1f87bb4f4387cffdf35a21a5e5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer faec855a53e74875a88bbb0c1a5107a6 at {username='oryx', principal='oryx/127.23.194.61@KRBTEST.COM'} at 127.0.0.1:56884: session id = faec855a53e74875a88bbb0c1a5107a6-00000000000000000000000000000000
I20260808 23:00:10.841184 1466 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 23:00:10.841811 2133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 23:00:10.842832 2133 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6: tablet copy: Starting download of 0 data blocks...
I20260808 23:00:10.842967 2133 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6: tablet copy: Starting download of 1 WAL segments...
I20260808 23:00:10.843976 2133 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 23:00:10.844805 1466 tablet_copy_service.cc:342] P bc466d1f87bb4f4387cffdf35a21a5e5: Request end of tablet copy session faec855a53e74875a88bbb0c1a5107a6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.23.194.61@KRBTEST.COM'} at 127.0.0.1:56884
I20260808 23:00:10.844872 1466 tablet_copy_service.cc:433] P bc466d1f87bb4f4387cffdf35a21a5e5: ending tablet copy session faec855a53e74875a88bbb0c1a5107a6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer faec855a53e74875a88bbb0c1a5107a6
I20260808 23:00:10.847164 2133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:10.848233 2311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:10.848354 2309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:10.848555 2133 server_base.cc:1034] running on GCE node
W20260808 23:00:10.849505 2316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:10.849653 2133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:10.849787 2133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:10.850898 2133 hybrid_clock.cc:648] HybridClock initialized: now 1786230010850892 us; error 24 us; skew 500 ppm
Aug 08 23:00:10 dist-test-slave-f0tn krb5kdc[1363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230010, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:10.852198 2133 init.cc:377] Logged in from keytab as oryx/127.23.194.61@KRBTEST.COM (short username oryx)
I20260808 23:00:10.852451 2133 webserver.cc:460] Webserver started at http://127.23.194.61:37413/ using document root <none> and password file <none>
I20260808 23:00:10.852568 2133 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:10.852599 2133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:10.853130 2133 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:10.853614 2324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:10.853742 2133 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:10.853785 2133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/wal
uuid: "faec855a53e74875a88bbb0c1a5107a6"
format_stamp: "Formatted at 2026-08-08 23:00:10 on dist-test-slave-f0tn"
server_key: "a6ea22897651aed6b34392fb90118a5b"
server_key_iv: "9d44637108e6b6bbbcd2ce245eb71c24"
server_key_version: "encryptionkey@0"
I20260808 23:00:10.853839 2133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:10.877851 2133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:10.879388 2133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:10.882254 2133 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.61:38237
I20260808 23:00:10.882272 2394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.61:38237 every 8 connection(s)
I20260808 23:00:10.882570 2133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-1/data/info.pb
I20260808 23:00:10.883502 2395 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:00:10.884135 2395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6: Bootstrap starting.
I20260808 23:00:10.887326 2395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:00:10.887550 2395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6: Bootstrap complete.
I20260808 23:00:10.887743 2395 raft_consensus.cc:348] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: NON_VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: true } }
I20260808 23:00:10.887828 2395 raft_consensus.cc:729] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: faec855a53e74875a88bbb0c1a5107a6, State: Initialized, Role: LEARNER
I20260808 23:00:10.887867 2395 consensus_queue.cc:260] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: NON_VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: true } }
I20260808 23:00:10.888043 2395 sys_catalog.cc:564] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:00:10.889341 2403 sys_catalog.cc:455] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: NON_VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: true } } }
I20260808 23:00:10.889460 2403 sys_catalog.cc:458] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [sys.catalog]: This master's current role is: LEARNER
I20260808 23:00:10.889698 2421 catalog_manager.cc:1259] Loaded cluster ID: ad6ba00deee14b4ea2be0cd0685da656
I20260808 23:00:10.889817 2421 catalog_manager.cc:1552] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6: loading cluster ID for follower catalog manager: success
I20260808 23:00:10.891077 2421 catalog_manager.cc:1574] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6: acquiring CA information for follower catalog manager: success
I20260808 23:00:10.891367 2398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:10.883822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36048 (local address 127.23.194.61:38237)
0808 23:00:10.883951 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:10.883953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:10.883960 (+ 7us) server_negotiation.cc:407] Connection header received
0808 23:00:10.883982 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:10.883983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:10.883992 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:10.884053 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:10.884594 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.885242 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.886078 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:10.886283 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:10.886338 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:10.887448 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:10.887450 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:10.887450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:10.887458 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:10.889641 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.890289 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.890291 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.890291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.890360 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:10.890549 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:10.890550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:10.890550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:10.890656 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:10.890735 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:10.891184 (+ 449us) server_negotiation.cc:299] Negotiation successful
0808 23:00:10.891280 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
I20260808 23:00:10.891393 2421 catalog_manager.cc:1602] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
WARNING: no policy specified for oryx/127.23.194.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.194.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.194.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.60.keytab.
Entry for principal oryx/127.23.194.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.60.keytab.
I20260808 23:00:10.915359 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.60:46459
--webserver_interface=127.23.194.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.60.keytab
--principal=oryx/127.23.194.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:35945,127.23.194.61:38237,127.23.194.60:46459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:10.997370 2424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:10.997601 2424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:10.997627 2424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:10.998797 2424 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:00:10.998845 2424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:10.998858 2424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:10.998881 2424 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:00:10.998904 2424 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:00:11.000445 2424 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:35945,127.23.194.61:38237,127.23.194.60:46459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.60.keytab
--principal=oryx/127.23.194.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.60:46459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.194.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:11.000690 2424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:11.000907 2424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:11.005650 2434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:11.005689 2433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:11.005677 2437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:11.005899 2424 server_base.cc:1034] running on GCE node
I20260808 23:00:11.006116 2424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:11.006412 2424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:11.007550 2424 hybrid_clock.cc:648] HybridClock initialized: now 1786230011007540 us; error 27 us; skew 500 ppm
Aug 08 23:00:11 dist-test-slave-f0tn krb5kdc[1363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230011, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:11.010010 2424 init.cc:377] Logged in from keytab as oryx/127.23.194.60@KRBTEST.COM (short username oryx)
I20260808 23:00:11.010468 2424 webserver.cc:460] Webserver started at http://127.23.194.60:35357/ using document root <none> and password file <none>
I20260808 23:00:11.010695 2424 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:11.010739 2424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:11.010860 2424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:11.011866 2424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/data/instance:
uuid: "8fa090f9309c484abd2d50c073281126"
format_stamp: "Formatted at 2026-08-08 23:00:11 on dist-test-slave-f0tn"
server_key: "4c14d03918ca43e86a8efd102a5d849d"
server_key_iv: "ebdb5c86418daf3f12fada5b7aa28b56"
server_key_version: "encryptionkey@0"
I20260808 23:00:11.012212 2424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/wal/instance:
uuid: "8fa090f9309c484abd2d50c073281126"
format_stamp: "Formatted at 2026-08-08 23:00:11 on dist-test-slave-f0tn"
server_key: "4c14d03918ca43e86a8efd102a5d849d"
server_key_iv: "ebdb5c86418daf3f12fada5b7aa28b56"
server_key_version: "encryptionkey@0"
I20260808 23:00:11.013500 2424 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 23:00:11.014201 2445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:11.014330 2424 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:11.014402 2424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/wal
uuid: "8fa090f9309c484abd2d50c073281126"
format_stamp: "Formatted at 2026-08-08 23:00:11 on dist-test-slave-f0tn"
server_key: "4c14d03918ca43e86a8efd102a5d849d"
server_key_iv: "ebdb5c86418daf3f12fada5b7aa28b56"
server_key_version: "encryptionkey@0"
I20260808 23:00:11.014467 2424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:11.025714 2424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:11.027784 2424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:11.027913 2424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:11.037611 2424 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.60:46459
I20260808 23:00:11.037693 2531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.60:46459 every 8 connection(s)
I20260808 23:00:11.038100 2424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/data/info.pb
I20260808 23:00:11.039271 2532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:11.040799 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 2424
I20260808 23:00:11.041950 2532 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:00:11 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.194.60@KRBTEST.COM
Aug 08 23:00:11 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230011, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.60@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM
I20260808 23:00:11.052335 1513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.043443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56894 (local address 127.23.194.62:35945)
0808 23:00:11.043533 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.043534 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.043544 (+ 10us) server_negotiation.cc:407] Connection header received
0808 23:00:11.043587 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.043587 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.043595 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.043643 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.044633 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.045325 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.046596 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.046840 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.046891 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.049668 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.049670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.049670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.049681 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.051157 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.051527 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.051528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.051529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.051596 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.051772 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.051773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.051774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.051895 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.051958 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.052135 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.052236 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 23:00:11.052377 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.041510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38962 (local address 127.23.194.60:46459)
0808 23:00:11.041759 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.041767 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.041807 (+ 40us) server_negotiation.cc:407] Connection header received
0808 23:00:11.042525 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.042533 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.042586 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.042705 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.043422 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.044845 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.045767 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.046401 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.046515 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.048983 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.048993 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.048993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.049001 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.050913 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.051269 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.051273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.051274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.051361 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.051535 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.051536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.051537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.051768 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.051900 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.052006 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.052129 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":73,"threads_started":1}
I20260808 23:00:11.052738 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.043177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.0.0.1:56894)
0808 23:00:11.043440 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.043477 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.043533 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.043751 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.043753 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.044274 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:11.044528 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.044534 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.045555 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.045556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.046516 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.046518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.047015 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.047017 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.047144 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.047635 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:11.047644 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:11.049575 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:11.051229 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.051235 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.051238 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.051434 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.051655 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.051656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.051656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.051717 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.051956 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:11.051963 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:11.052018 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.052099 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.052322 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":90,"threads_started":1}
I20260808 23:00:11.053272 2532 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:00:11 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230011, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.60@KRBTEST.COM for oryx/127.23.194.61@KRBTEST.COM
I20260808 23:00:11.059216 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.053543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:38237 (local address 127.0.0.1:36056)
0808 23:00:11.053600 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.053612 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.053654 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.053781 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.053781 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.053886 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:11.054022 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.054025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.054658 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.054659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.055164 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.055165 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.055523 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.055524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.055609 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.055945 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:11.055949 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:11.057067 (+ 1118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:11.058535 (+ 1468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.058535 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.058536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.058674 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.058812 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.058813 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.058813 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.058856 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.059016 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:11.059016 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:11.059046 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.059088 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.059157 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 23:00:11.059244 2398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.053628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36056 (local address 127.23.194.61:38237)
0808 23:00:11.053673 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.053675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.053684 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:00:11.053711 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.053712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.053724 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.053779 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.054099 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.054587 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.055250 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.055458 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.055515 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.057148 (+ 1633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.057149 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.057149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.057164 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.058477 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.058725 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.058726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.058726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.058772 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.058889 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.058889 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.058889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.058979 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.059021 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.059105 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.059182 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 23:00:11.059630 2532 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:00:11 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230011, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.60@KRBTEST.COM for oryx/127.23.194.60@KRBTEST.COM
I20260808 23:00:11.065239 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.059862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:46459 (local address 127.0.0.1:38974)
0808 23:00:11.059902 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.059912 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.059947 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.060094 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.060095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.060209 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:11.060236 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.060237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.060876 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.060876 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.061338 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.061339 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.061709 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.061709 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.061758 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.062047 (+ 289us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:11.062050 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:11.063049 (+ 999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:11.064416 (+ 1367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.064417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.064418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.064567 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.064790 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.064790 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.064791 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.064834 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.065031 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:11.065032 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:11.065063 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.065107 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.065179 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 23:00:11.065305 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.059916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38974 (local address 127.23.194.60:46459)
0808 23:00:11.059948 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.059950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.059974 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:00:11.059990 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.059991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.059999 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.060032 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.060294 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.060818 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.061453 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.061646 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.061705 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.063143 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.063145 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.063145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.063151 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.064345 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.064659 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.064660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.064661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.064733 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.064873 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.064873 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.064873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.064969 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.065038 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.065127 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.065228 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":16}
I20260808 23:00:11.066953 2532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: Bootstrap starting.
I20260808 23:00:11.068320 2532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:11.068936 2532 log.cc:824] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:11.069777 2532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: No bootstrap required, opened a new log
I20260808 23:00:11.072551 2532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } }
I20260808 23:00:11.072690 2532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:11.072721 2532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fa090f9309c484abd2d50c073281126, State: Initialized, Role: FOLLOWER
I20260808 23:00:11.072840 2532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } }
I20260808 23:00:11.073233 2532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:00:11.073262 2539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } } }
I20260808 23:00:11.073351 2539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 23:00:11.075430 2554 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:00:11.075490 2554 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:00:11.077924 2424 master_runner.cc:428] Detected that this master 8fa090f9309c484abd2d50c073281126 is joining an existing cluster
I20260808 23:00:11.077967 2424 master_runner.cc:432] Initiating AddMaster RPC to add 127.23.194.60:46459
I20260808 23:00:11.086683 2398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.079230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36070 (local address 127.23.194.61:38237)
0808 23:00:11.079278 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.079281 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.079860 (+ 579us) server_negotiation.cc:407] Connection header received
0808 23:00:11.079966 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.079968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.079979 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.080010 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.080728 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.081216 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.082221 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.082421 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.082475 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.083667 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.083669 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.083669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.083676 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.085192 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.085810 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.085812 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.085812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.085878 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.086100 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.086101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.086101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.086244 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.086291 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.086500 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.086583 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 23:00:11.087520 2562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.079239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:38237 (local address 127.0.0.1:36070)
0808 23:00:11.079797 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.079809 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.079888 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.080131 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.080132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.080469 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:11.080650 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.080652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.081288 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.081288 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.082135 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.082138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.082533 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.082534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.082623 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.083084 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:11.083088 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:11.083598 (+ 510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:11.085262 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.085263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.085264 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.085729 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.085957 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.085957 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.085958 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.086023 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.086346 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:11.086346 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:11.086389 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.086904 (+ 515us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.087005 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":677,"threads_started":1}
I20260808 23:00:11.088264 1513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.079208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56898 (local address 127.23.194.62:35945)
0808 23:00:11.079256 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.079258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.080290 (+ 1032us) server_negotiation.cc:407] Connection header received
0808 23:00:11.080348 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.080349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.080362 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.080420 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.081077 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.081862 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.083179 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.083461 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.083535 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.084716 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.084717 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.084717 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.084725 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.086322 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.086718 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.086719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.086719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.086787 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.087288 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.087289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.087290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.087427 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.087477 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.088036 (+ 559us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.088117 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 23:00:11.088652 2567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.079425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.0.0.1:56898)
0808 23:00:11.080207 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.080217 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.080287 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.080424 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.080425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.080770 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:11.080968 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.080971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.081959 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.081960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.083088 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.083091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.083541 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.083541 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.083650 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.084092 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:11.084096 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:11.084590 (+ 494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:11.086407 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.086408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.086409 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.086610 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.087126 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.087127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.087127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.087210 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.087530 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:11.087530 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:11.087875 (+ 345us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.088440 (+ 565us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.088545 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":127,"threads_started":1}
Aug 08 23:00:11 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230010, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.62@KRBTEST.COM for oryx/127.23.194.60@KRBTEST.COM
I20260808 23:00:11.103932 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 1534
I20260808 23:00:11.118418 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.079851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38976 (local address 127.23.194.60:46459)
0808 23:00:11.079885 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.079886 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.085588 (+ 5702us) server_negotiation.cc:407] Connection header received
0808 23:00:11.085681 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.085681 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.085690 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.085717 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.086453 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.087059 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.088180 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.088423 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.088473 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.114899 (+ 26426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.114901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.114901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.114909 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.116714 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.117175 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.117176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.117176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.117227 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.117683 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.117684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.117684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.117820 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.117874 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.118237 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.118307 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 23:00:11.118448 2563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.079989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:46459 (local address 127.0.0.1:38976)
0808 23:00:11.085524 (+ 5535us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.085535 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.085610 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.085768 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.085770 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.086174 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:11.086377 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.086378 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.087151 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.087151 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.088119 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.088122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.088619 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.088619 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.088754 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.089478 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:11.089483 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:11.114660 (+ 25177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:11.116823 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.116824 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.116825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.117094 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.117543 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.117544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.117544 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.117615 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.117893 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:11.117894 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:11.117927 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.118137 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.118249 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5498,"thread_start_us":130,"threads_started":1}
I20260808 23:00:11.120450 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:36397
--local_ip_for_outbound_sockets=127.23.194.1
--tserver_master_addrs=127.23.194.62:35945
--webserver_port=40025
--webserver_interface=127.23.194.1
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.1.keytab
--principal=oryx/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.194.62:35945,127.23.194.61:38237
--tserver_master_addrs=127.23.194.62:35945,127.23.194.61:38237,127.23.194.60:46459 with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:00:11.120584 2284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.090646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:46459 (local address 127.0.0.1:38992)
0808 23:00:11.090703 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.090712 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.090744 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.091097 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.091099 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.091241 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:11.091278 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.091279 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.091989 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.091990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.092488 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.092489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.092863 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.092864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.092931 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.093295 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:11.093298 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:11.094592 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:11.119345 (+ 24753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.119346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.119347 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.119587 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.119816 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.119817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.119817 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.119881 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.120197 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:11.120199 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:11.120265 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.120354 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.120487 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 23:00:11.121001 2568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.090705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38992 (local address 127.23.194.60:46459)
0808 23:00:11.090975 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.090976 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.090982 (+ 6us) server_negotiation.cc:407] Connection header received
0808 23:00:11.091011 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.091011 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.091019 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.091074 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.091343 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.091915 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.092561 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.092797 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.092851 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.094761 (+ 1910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.094763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.094763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.094769 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.119250 (+ 24481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.119672 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.119673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.119674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.119744 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.119955 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.119956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.119956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.120115 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.120183 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.120389 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.120471 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":94,"threads_started":1}
I20260808 23:00:11.121668 1431 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.23.194.60:46459: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8fa090f9309c484abd2d50c073281126"
member_type: NON_VOTER
last_known_addr {
host: "127.23.194.60"
port: 46459
}
attrs {
promote: true
}
}
dest_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5"
cas_config_opid_index: 5
I20260808 23:00:11.121979 1431 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: NON_VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: true } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: NON_VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: true } }
I20260808 23:00:11.122408 2286 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8fa090f9309c484abd2d50c073281126 (127.23.194.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: NON_VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: true } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: NON_VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: true } } }
I20260808 23:00:11.123062 2489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:11.123433 2286 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.23.194.60:46459
I20260808 23:00:11.123512 2286 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: NON_VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: true } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: NON_VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: true } } }
I20260808 23:00:11.123577 2286 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:11.124063 2489 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 1 FOLLOWER]: Refusing update from remote peer bc466d1f87bb4f4387cffdf35a21a5e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 23:00:11.124305 2424 master.cc:537] Master@127.23.194.60:46459 shutting down...
I20260808 23:00:11.124344 2285 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: NON_VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 23:00:11.125048 1402 consensus_peers.cc:487] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 -> Peer 8fa090f9309c484abd2d50c073281126 (127.23.194.60:46459): Couldn't send request to peer 8fa090f9309c484abd2d50c073281126. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 23:00:11.128079 2424 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:00:11.128189 2424 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:11.128252 2424 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: stopping tablet replica
I20260808 23:00:11.129684 2284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.122562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:38237 (local address 127.0.0.1:36078)
0808 23:00:11.125561 (+ 2999us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.125573 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.125660 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.125887 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.125888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.125889 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:11.125936 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.125938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.126867 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.126868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.128342 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.128346 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.129414 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.129415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.129485 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.129511 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.129571 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.129613 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2968}
I20260808 23:00:11.129716 2398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.122689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36078 (local address 127.23.194.61:38237)
0808 23:00:11.122737 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.122739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.125628 (+ 2889us) server_negotiation.cc:407] Connection header received
0808 23:00:11.125733 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.125734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.125762 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.125802 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:11.126012 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.126764 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.128441 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.129058 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.129104 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.129172 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.129588 (+ 416us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.129644 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 23:00:11.130448 2364 raft_consensus.cc:1261] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 1 LEARNER]: Refusing update from remote peer bc466d1f87bb4f4387cffdf35a21a5e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 23:00:11.130836 2285 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: NON_VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 23:00:11.131470 2364 raft_consensus.cc:2932] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8fa090f9309c484abd2d50c073281126 (127.23.194.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: NON_VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: true } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: NON_VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: true } } }
I20260808 23:00:11.132568 2403 sys_catalog.cc:455] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc466d1f87bb4f4387cffdf35a21a5e5. Latest consensus state: current_term: 1 leader_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: NON_VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: true } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: NON_VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: true } } }
I20260808 23:00:11.132650 2403 sys_catalog.cc:458] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [sys.catalog]: This master's current role is: LEARNER
I20260808 23:00:11.132591 2571 sys_catalog.cc:455] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: NON_VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: true } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: NON_VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: true } } }
I20260808 23:00:11.132701 2571 sys_catalog.cc:458] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [sys.catalog]: This master's current role is: LEARNER
I20260808 23:00:11.133613 2285 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5: attempting to promote NON_VOTER faec855a53e74875a88bbb0c1a5107a6 to VOTER
I20260808 23:00:11.133800 2285 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: NON_VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: true } }
I20260808 23:00:11.134374 2284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.134164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:46459 (local address 127.0.0.1:38998)
0808 23:00:11.134295 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.134345 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:46459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260808 23:00:11.134380 2364 raft_consensus.cc:1261] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 1 LEARNER]: Refusing update from remote peer bc466d1f87bb4f4387cffdf35a21a5e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 23:00:11.134660 2285 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 23:00:11.134718 2284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.134628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:46459 (local address 127.0.0.1:39004)
0808 23:00:11.134667 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.134691 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:46459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 23:00:11.134783 1402 consensus_peers.cc:487] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 -> Peer 8fa090f9309c484abd2d50c073281126 (127.23.194.60:46459): Couldn't send request to peer 8fa090f9309c484abd2d50c073281126. Status: Network error: Client connection negotiation failed: client connection to 127.23.194.60:46459: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 23:00:11.135311 2285 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, faec855a53e74875a88bbb0c1a5107a6 (127.23.194.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: NON_VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: true } } }
I20260808 23:00:11.136199 2285 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: NON_VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: true } } }
I20260808 23:00:11.136270 2285 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:11.136240 2288 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: NON_VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: true } } }
I20260808 23:00:11.136337 2288 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:11.136652 2364 raft_consensus.cc:2932] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, faec855a53e74875a88bbb0c1a5107a6 (127.23.194.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: NON_VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: true } } }
I20260808 23:00:11.137861 2571 sys_catalog.cc:455] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: NON_VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: true } } }
I20260808 23:00:11.137944 2571 sys_catalog.cc:458] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:00:11.144708 2424 master.cc:552] Master@127.23.194.60:46459 shutdown complete.
I20260808 23:00:11.144773 2424 master_runner.cc:305] Clearing existing system tablet
I20260808 23:00:11.144910 2424 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:11.144959 2424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:11.145720 2424 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 23:00:11.146248 2574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:11.146405 2424 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:11.146456 2424 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:00:11.146479 2424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/wal
uuid: "8fa090f9309c484abd2d50c073281126"
format_stamp: "Formatted at 2026-08-08 23:00:11 on dist-test-slave-f0tn"
server_key: "4c14d03918ca43e86a8efd102a5d849d"
server_key_iv: "ebdb5c86418daf3f12fada5b7aa28b56"
server_key_version: "encryptionkey@0"
I20260808 23:00:11.147254 2424 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:11.148691 2424 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 23:00:11.148762 2424 log.cc:1196] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 23:00:11.148947 2424 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: Deleting consensus metadata
I20260808 23:00:11.149065 2424 master_runner.cc:315] Copying system tablet from 127.23.194.62:35945
I20260808 23:00:11.149690 2424 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: tablet copy: Beginning tablet copy session from remote peer at address 127.23.194.62:35945
I20260808 23:00:11.160017 2583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.150068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.0.0.1:56914)
0808 23:00:11.153233 (+ 3165us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.153248 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.153337 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.153526 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.153526 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.153737 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:11.153799 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.153802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.154752 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.154753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.155573 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.155575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.156038 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.156039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.156096 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.156613 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:11.156619 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:11.157013 (+ 394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:11.158873 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.158874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.158875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.159097 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.159317 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.159319 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.159319 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.159388 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.159671 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:11.159672 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:11.159730 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.159807 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.159916 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3130,"thread_start_us":78,"threads_started":1}
I20260808 23:00:11.160020 1513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.150074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56914 (local address 127.23.194.62:35945)
0808 23:00:11.150133 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.150136 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.153338 (+ 3202us) server_negotiation.cc:407] Connection header received
0808 23:00:11.153422 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.153423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.153438 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.153514 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.153886 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.154669 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.155659 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.155962 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.156033 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.157110 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.157111 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.157111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.157119 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.158783 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.159177 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.159179 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.159179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.159249 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.159450 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.159452 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.159452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.159604 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.159678 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.159844 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.159927 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 23:00:11.160748 1466 tablet_copy_service.cc:140] P bc466d1f87bb4f4387cffdf35a21a5e5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8fa090f9309c484abd2d50c073281126 ({username='oryx', principal='oryx/127.23.194.60@KRBTEST.COM'} at 127.0.0.1:56914)
I20260808 23:00:11.160821 1466 tablet_copy_service.cc:161] P bc466d1f87bb4f4387cffdf35a21a5e5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8fa090f9309c484abd2d50c073281126 at {username='oryx', principal='oryx/127.23.194.60@KRBTEST.COM'} at 127.0.0.1:56914: session id = 8fa090f9309c484abd2d50c073281126-00000000000000000000000000000000
I20260808 23:00:11.161507 1466 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 23:00:11.162050 2424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 23:00:11.163362 2424 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: tablet copy: Starting download of 0 data blocks...
I20260808 23:00:11.163508 2424 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: tablet copy: Starting download of 1 WAL segments...
I20260808 23:00:11.164247 2424 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 23:00:11.165120 1466 tablet_copy_service.cc:342] P bc466d1f87bb4f4387cffdf35a21a5e5: Request end of tablet copy session 8fa090f9309c484abd2d50c073281126-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.23.194.60@KRBTEST.COM'} at 127.0.0.1:56914
I20260808 23:00:11.165171 1466 tablet_copy_service.cc:433] P bc466d1f87bb4f4387cffdf35a21a5e5: ending tablet copy session 8fa090f9309c484abd2d50c073281126-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8fa090f9309c484abd2d50c073281126
I20260808 23:00:11.168136 2424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:11.169330 2584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:11.169345 2588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:11.169781 2424 server_base.cc:1034] running on GCE node
W20260808 23:00:11.169817 2585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:11.170079 2424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:11.170256 2424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:11.171392 2424 hybrid_clock.cc:648] HybridClock initialized: now 1786230011171377 us; error 35 us; skew 500 ppm
Aug 08 23:00:11 dist-test-slave-f0tn krb5kdc[1363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230011, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:11.172896 2424 init.cc:377] Logged in from keytab as oryx/127.23.194.60@KRBTEST.COM (short username oryx)
I20260808 23:00:11.173166 2424 webserver.cc:460] Webserver started at http://127.23.194.60:40715/ using document root <none> and password file <none>
I20260808 23:00:11.173288 2424 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:11.173323 2424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:11.174081 2424 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:00:11.174667 2600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:11.174834 2424 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:00:11.174886 2424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/wal
uuid: "8fa090f9309c484abd2d50c073281126"
format_stamp: "Formatted at 2026-08-08 23:00:11 on dist-test-slave-f0tn"
server_key: "4c14d03918ca43e86a8efd102a5d849d"
server_key_iv: "ebdb5c86418daf3f12fada5b7aa28b56"
server_key_version: "encryptionkey@0"
I20260808 23:00:11.174945 2424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:11.185557 2424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:11.187428 2424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:11.192744 2424 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.60:46459
I20260808 23:00:11.193070 2424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/master-2/data/info.pb
I20260808 23:00:11.194410 2676 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:00:11.194882 2676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: Bootstrap starting.
I20260808 23:00:11.196244 2673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.60:46459 every 8 connection(s)
I20260808 23:00:11.197933 2676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:00:11.198140 2676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: Bootstrap complete.
I20260808 23:00:11.198349 2676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: NON_VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: true } }
I20260808 23:00:11.198441 2676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8fa090f9309c484abd2d50c073281126, State: Initialized, Role: LEARNER
I20260808 23:00:11.198474 2676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: NON_VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: true } }
I20260808 23:00:11.198691 2676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:00:11.198673 2682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: NON_VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: true } } }
I20260808 23:00:11.198905 2682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [sys.catalog]: This master's current role is: LEARNER
I20260808 23:00:11.200191 2697 catalog_manager.cc:1259] Loaded cluster ID: ad6ba00deee14b4ea2be0cd0685da656
I20260808 23:00:11.200284 2697 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: loading cluster ID for follower catalog manager: success
I20260808 23:00:11.202023 2697 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: acquiring CA information for follower catalog manager: success
I20260808 23:00:11.202342 2697 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 23:00:11.226817 2569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:11.227033 2569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:11.227058 2569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:11.228272 2569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:11.228317 2569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:11.228344 2569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:00:11.230122 2569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.1.keytab
--principal=oryx/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:36397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=40025
--tserver_master_addrs=127.23.194.62:35945,127.23.194.61:38237,127.23.194.60:46459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:11.230377 2569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:11.230598 2569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:11.237287 2711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:11.237370 2708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:11.237689 2569 server_base.cc:1034] running on GCE node
W20260808 23:00:11.237820 2706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:11.238044 2569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:11.238288 2569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:11.239427 2569 hybrid_clock.cc:648] HybridClock initialized: now 1786230011239416 us; error 30 us; skew 500 ppm
Aug 08 23:00:11 dist-test-slave-f0tn krb5kdc[1363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230011, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:11.242287 2569 init.cc:377] Logged in from keytab as oryx/127.23.194.1@KRBTEST.COM (short username oryx)
I20260808 23:00:11.242743 2569 webserver.cc:460] Webserver started at http://127.23.194.1:40025/ using document root <none> and password file <none>
I20260808 23:00:11.242954 2569 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:11.243021 2569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:11.244489 2569 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:00:11.245056 2720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:11.245198 2569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:11.245283 2569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "330d9fc4eb7b4a089b33ab13a463351c"
format_stamp: "Formatted at 2026-08-08 23:00:10 on dist-test-slave-f0tn"
server_key: "a8d9852da0a63a9c2ecfd6ac0fcb1ec2"
server_key_iv: "feb94d2d64bf4fd125ce5acebe726081"
server_key_version: "encryptionkey@0"
I20260808 23:00:11.245641 2569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:11.259586 2569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:11.261970 2569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:11.262151 2569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:11.262517 2569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:11.262874 2569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:11.262912 2569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:11.262943 2569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:11.262967 2569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:11.277076 2569 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:36397
I20260808 23:00:11.277177 2890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:36397 every 8 connection(s)
I20260808 23:00:11.277597 2569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:00:11.278609 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 2569
I20260808 23:00:11.279036 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 1739
I20260808 23:00:11.292801 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:42459
--local_ip_for_outbound_sockets=127.23.194.2
--tserver_master_addrs=127.23.194.62:35945
--webserver_port=33649
--webserver_interface=127.23.194.2
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.2.keytab
--principal=oryx/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.194.62:35945,127.23.194.61:38237
--tserver_master_addrs=127.23.194.62:35945,127.23.194.61:38237,127.23.194.60:46459 with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:00:11 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230011, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM
Aug 08 23:00:11 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230011, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.61@KRBTEST.COM
Aug 08 23:00:11 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230011, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.60@KRBTEST.COM
I20260808 23:00:11.298624 2398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.280386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:59285 (local address 127.23.194.61:38237)
0808 23:00:11.280450 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.280453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.287875 (+ 7422us) server_negotiation.cc:407] Connection header received
0808 23:00:11.287984 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.287985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.287997 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.288026 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.289303 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.290103 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.291879 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.292196 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.292234 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.295581 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.295582 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.295583 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.295592 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.297060 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.297540 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.297541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.297541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.297605 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.297847 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.297847 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.297848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.297987 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.298039 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.298442 (+ 403us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.298527 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 23:00:11.298678 1513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.278566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:51787 (local address 127.23.194.62:35945)
0808 23:00:11.278622 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.278624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.287321 (+ 8697us) server_negotiation.cc:407] Connection header received
0808 23:00:11.287372 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.287374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.287388 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.287430 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.289307 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.290139 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.291886 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.292158 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.292198 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.295285 (+ 3087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.295286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.295287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.295295 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.296743 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.297502 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.297503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.297503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.297567 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.297839 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.297840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.297840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.297968 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.298023 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.298479 (+ 456us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.298563 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 23:00:11.299083 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.278442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:46459 (local address 127.23.194.1:55493)
0808 23:00:11.278923 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.278970 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.279748 (+ 778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.288057 (+ 8309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.288058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.288766 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:11.289182 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.289191 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.290415 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.290416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.291773 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.291775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.292297 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.292298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.292433 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.293307 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:11.293312 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:11.296134 (+ 2822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:11.297814 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.297814 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.297815 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.297994 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.298190 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.298191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.298191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.298244 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.298518 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:11.298519 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:11.298553 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.298620 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.298817 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":106,"thread_start_us":172,"threads_started":1}
I20260808 23:00:11.299086 2896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.278442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.23.194.1:51787)
0808 23:00:11.287183 (+ 8741us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.287195 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.287271 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.287666 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.287671 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.288822 (+ 1151us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:11.289182 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.289190 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.290383 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.290384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.291774 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.291777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.292291 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.292292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.292430 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.293167 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:11.293173 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:11.295164 (+ 1991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:11.296814 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.296822 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.296825 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.297417 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.297632 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.297633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.297633 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.297769 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.298062 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:11.298070 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:11.298238 (+ 168us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.298839 (+ 601us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.298984 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8698,"mutex_wait_us":403,"thread_start_us":101,"threads_started":1}
I20260808 23:00:11.299530 2899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.287727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:55493 (local address 127.23.194.60:46459)
0808 23:00:11.287842 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.287844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.287852 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:00:11.287879 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.287880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.287892 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.288012 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.289307 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.290326 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.291873 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.292190 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.292246 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.296234 (+ 3988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.296235 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.296236 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.296244 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.297738 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.298067 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.298068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.298068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.298128 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.298304 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.298305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.298305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.298439 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.298484 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.299349 (+ 865us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.299437 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":58,"threads_started":1}
I20260808 23:00:11.299655 2893 heartbeater.cc:347] Connected to a master server at 127.23.194.62:35945
I20260808 23:00:11.299754 2893 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:11.299841 2891 heartbeater.cc:347] Connected to a master server at 127.23.194.60:46459
I20260808 23:00:11.299881 2891 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:11.300004 2893 heartbeater.cc:511] Master 127.23.194.62:35945 requested a full tablet report, sending...
I20260808 23:00:11.300004 2891 heartbeater.cc:511] Master 127.23.194.60:46459 requested a full tablet report, sending...
I20260808 23:00:11.300302 2898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.278820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:38237 (local address 127.23.194.1:59285)
0808 23:00:11.287788 (+ 8968us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.287801 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.287905 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.288084 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.288085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.288752 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:11.289182 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.289191 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.290337 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.290339 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.291772 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.291775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.292311 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.292311 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.292433 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.293144 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:11.293165 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:11.295476 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:11.297145 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.297146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.297146 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.297451 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.297685 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.297686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.297686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.297773 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.298069 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:11.298077 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:11.298238 (+ 161us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.299083 (+ 845us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.299748 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8926,"mutex_wait_us":111,"thread_start_us":8199,"threads_started":1}
I20260808 23:00:11.300565 2624 ts_manager.cc:194] Registered new tserver with Master: 330d9fc4eb7b4a089b33ab13a463351c (127.23.194.1:36397)
I20260808 23:00:11.300715 2894 heartbeater.cc:347] Connected to a master server at 127.23.194.61:38237
I20260808 23:00:11.300755 2894 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:11.300854 2894 heartbeater.cc:511] Master 127.23.194.61:38237 requested a full tablet report, sending...
I20260808 23:00:11.300917 1431 ts_manager.cc:194] Re-registered known tserver with Master: 330d9fc4eb7b4a089b33ab13a463351c (127.23.194.1:36397)
I20260808 23:00:11.301256 2351 ts_manager.cc:194] Registered new tserver with Master: 330d9fc4eb7b4a089b33ab13a463351c (127.23.194.1:36397)
I20260808 23:00:11.301291 1431 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:51787
W20260808 23:00:11.379053 2901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:11.379330 2901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:11.379354 2901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:11.380543 2901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:11.380587 2901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:11.380614 2901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:00:11.382179 2901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.2.keytab
--principal=oryx/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:42459
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=33649
--tserver_master_addrs=127.23.194.62:35945,127.23.194.61:38237,127.23.194.60:46459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:11.382432 2901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:11.382650 2901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:11.387638 2914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:11.387662 2912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:11.387732 2911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:11.387892 2901 server_base.cc:1034] running on GCE node
I20260808 23:00:11.388098 2901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:11.388352 2901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:11.389505 2901 hybrid_clock.cc:648] HybridClock initialized: now 1786230011389492 us; error 28 us; skew 500 ppm
Aug 08 23:00:11 dist-test-slave-f0tn krb5kdc[1363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230011, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:11.391860 2901 init.cc:377] Logged in from keytab as oryx/127.23.194.2@KRBTEST.COM (short username oryx)
I20260808 23:00:11.392251 2901 webserver.cc:460] Webserver started at http://127.23.194.2:33649/ using document root <none> and password file <none>
I20260808 23:00:11.392460 2901 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:11.392525 2901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:11.393857 2901 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 23:00:11.394368 2925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:11.394513 2901 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:00:11.394593 2901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "c3b85349630b484dbd4ea782a8009075"
format_stamp: "Formatted at 2026-08-08 23:00:10 on dist-test-slave-f0tn"
server_key: "5e926782bbf85a1a61c3995d862ee384"
server_key_iv: "33d19dcd14621ed252310769200c5bae"
server_key_version: "encryptionkey@0"
I20260808 23:00:11.394912 2901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:11.405612 2901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:11.407297 2901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:11.407428 2901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:11.407627 2901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:11.407917 2901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:11.407951 2901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:11.407979 2901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:11.408013 2901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:11.413784 2901 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:42459
I20260808 23:00:11.413825 3076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:42459 every 8 connection(s)
I20260808 23:00:11.414160 2901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:00:11.420193 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 2901
I20260808 23:00:11.420385 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 1935
Aug 08 23:00:11 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230011, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.61@KRBTEST.COM
Aug 08 23:00:11 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230011, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM
Aug 08 23:00:11 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230011, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.60@KRBTEST.COM
I20260808 23:00:11.430009 2398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.415011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:37917 (local address 127.23.194.61:38237)
0808 23:00:11.415064 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.415066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.415318 (+ 252us) server_negotiation.cc:407] Connection header received
0808 23:00:11.415968 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.415969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.415980 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.416031 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.416782 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.417269 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.418589 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.418878 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.418932 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.422886 (+ 3954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.422887 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.422888 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.422894 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.427383 (+ 4489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.428587 (+ 1204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.428588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.428588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.428639 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.429534 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.429535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.429535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.429666 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.429702 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.429852 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.429924 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 23:00:11.430491 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.414806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:38237 (local address 127.23.194.2:37917)
0808 23:00:11.415185 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.415235 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.415815 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.416463 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.416464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.416621 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:11.416678 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.416680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.417366 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.417367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.418497 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.418500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.418943 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.418944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.419014 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.419646 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:11.419650 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:11.422244 (+ 2594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:11.428383 (+ 6139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.428384 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.428384 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.428522 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.428705 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.428706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.428706 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.428817 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.429738 (+ 921us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:11.429739 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:11.429771 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.430126 (+ 355us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.430273 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":112,"threads_started":1}
I20260808 23:00:11.430956 2899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.415003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:49047 (local address 127.23.194.60:46459)
0808 23:00:11.415070 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.415072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.415324 (+ 252us) server_negotiation.cc:407] Connection header received
0808 23:00:11.416082 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.416083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.416096 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.416138 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.416737 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.417537 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.418465 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.418722 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.418784 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.422873 (+ 4089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.422875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.422875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.422884 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.427967 (+ 5083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.428877 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.428878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.428878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.428923 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.429058 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.429058 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.429059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.429169 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.429203 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.430793 (+ 1590us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.430883 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 23:00:11.431186 3083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.414806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:46459 (local address 127.23.194.2:49047)
0808 23:00:11.415193 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.415234 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.415805 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.416178 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.416178 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.416464 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:11.416645 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.416650 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.417644 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.417645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.418381 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.418383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.418794 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.418794 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.419011 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.419631 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:11.419641 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:11.422548 (+ 2907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:11.428042 (+ 5494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.428054 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.428056 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.428324 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.428973 (+ 649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.428974 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.428974 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.429012 (+ 38us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.429235 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:11.429240 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:11.429417 (+ 177us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.431017 (+ 1600us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.431104 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":26,"thread_start_us":314,"threads_started":1}
I20260808 23:00:11.438313 3079 heartbeater.cc:347] Connected to a master server at 127.23.194.61:38237
I20260808 23:00:11.438455 3079 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:11.438731 3079 heartbeater.cc:511] Master 127.23.194.61:38237 requested a full tablet report, sending...
I20260808 23:00:11.439294 2351 ts_manager.cc:194] Registered new tserver with Master: c3b85349630b484dbd4ea782a8009075 (127.23.194.2:42459)
I20260808 23:00:11.441823 3077 heartbeater.cc:347] Connected to a master server at 127.23.194.60:46459
I20260808 23:00:11.441893 3077 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:11.441969 3077 heartbeater.cc:511] Master 127.23.194.60:46459 requested a full tablet report, sending...
I20260808 23:00:11.442317 2624 ts_manager.cc:194] Registered new tserver with Master: c3b85349630b484dbd4ea782a8009075 (127.23.194.2:42459)
I20260808 23:00:11.443791 1513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.414906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:51697 (local address 127.23.194.62:35945)
0808 23:00:11.414955 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.414957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.415318 (+ 361us) server_negotiation.cc:407] Connection header received
0808 23:00:11.415914 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.415915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.415928 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.415984 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.416767 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.417541 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.418644 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.418900 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.418969 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.422823 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.422825 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.422825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.422834 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.442541 (+ 19707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.442925 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.442927 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.442927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.442996 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.443217 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.443219 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.443219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.443384 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.443458 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.443617 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.443707 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 23:00:11.444037 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.414808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.23.194.2:51697)
0808 23:00:11.415237 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.415246 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.415813 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.416096 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.416100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.416519 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:11.416703 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.416705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.417609 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.417610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.418586 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.418588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.418997 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.418998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.419082 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.419718 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:11.419722 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:11.422753 (+ 3031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:11.442649 (+ 19896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.442650 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.442650 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.442861 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.443063 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.443064 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.443065 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.443122 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.443458 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:11.443459 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:11.443507 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.443567 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.443665 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":227,"thread_start_us":70,"threads_started":1}
I20260808 23:00:11.444487 3078 heartbeater.cc:347] Connected to a master server at 127.23.194.62:35945
I20260808 23:00:11.444533 3078 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:11.444602 3078 heartbeater.cc:511] Master 127.23.194.62:35945 requested a full tablet report, sending...
I20260808 23:00:11.444855 1431 ts_manager.cc:194] Re-registered known tserver with Master: c3b85349630b484dbd4ea782a8009075 (127.23.194.2:42459)
I20260808 23:00:11.445220 1431 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:51697
I20260808 23:00:11.447290 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:40999
--local_ip_for_outbound_sockets=127.23.194.3
--tserver_master_addrs=127.23.194.62:35945
--webserver_port=37483
--webserver_interface=127.23.194.3
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.3.keytab
--principal=oryx/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.194.62:35945,127.23.194.61:38237
--tserver_master_addrs=127.23.194.62:35945,127.23.194.61:38237,127.23.194.60:46459 with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:11.528133 3088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:11.528335 3088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:11.528358 3088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:11.529711 3088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:11.529768 3088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:11.529805 3088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:00:11.531615 3088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.3.keytab
--principal=oryx/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:40999
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=37483
--tserver_master_addrs=127.23.194.62:35945,127.23.194.61:38237,127.23.194.60:46459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:11.531949 3088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:11.532218 3088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 23:00:11.532260 2284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.529331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:46459 (local address 127.0.0.1:39012)
0808 23:00:11.529405 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.529418 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.529480 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.529646 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.529647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.529648 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:11.529698 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.529700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.530570 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.530570 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.531273 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.531275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.532039 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.532040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.532107 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.532111 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.532152 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.532202 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 23:00:11.532277 2899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.529421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39012 (local address 127.23.194.60:46459)
0808 23:00:11.529496 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.529499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.529511 (+ 12us) server_negotiation.cc:407] Connection header received
0808 23:00:11.529539 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.529540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.529564 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.529630 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:11.529758 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.530490 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.531362 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.531934 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.532031 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.532113 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.532175 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.532215 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 23:00:11.532644 2626 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 23:00:11.532851 2682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc466d1f87bb4f4387cffdf35a21a5e5. Latest consensus state: current_term: 1 leader_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: NON_VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: true } } }
I20260808 23:00:11.532943 2682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [sys.catalog]: This master's current role is: LEARNER
W20260808 23:00:11.537261 3098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:11.537266 3101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:11.537532 3088 server_base.cc:1034] running on GCE node
W20260808 23:00:11.537609 3096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:11.537797 3088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:11.538017 3088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:11.539158 3088 hybrid_clock.cc:648] HybridClock initialized: now 1786230011539140 us; error 29 us; skew 500 ppm
Aug 08 23:00:11 dist-test-slave-f0tn krb5kdc[1363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230011, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:11.542227 3088 init.cc:377] Logged in from keytab as oryx/127.23.194.3@KRBTEST.COM (short username oryx)
I20260808 23:00:11.542619 3088 webserver.cc:460] Webserver started at http://127.23.194.3:37483/ using document root <none> and password file <none>
I20260808 23:00:11.542837 3088 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:11.542889 3088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:11.544185 3088 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:00:11.544772 3115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:11.544968 3088 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:00:11.545058 3088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4"
format_stamp: "Formatted at 2026-08-08 23:00:10 on dist-test-slave-f0tn"
server_key: "80ff0c419d5e11468ed5c28cdcc49526"
server_key_iv: "ec6a40990bc96d248795e16fed042793"
server_key_version: "encryptionkey@0"
I20260808 23:00:11.545404 3088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:11.557721 3088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:11.559691 3088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:11.559796 3088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:11.559962 3088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:11.560236 3088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:11.560271 3088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:11.560292 3088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:11.560316 3088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:11.565871 3088 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:40999
I20260808 23:00:11.565950 3295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:40999 every 8 connection(s)
I20260808 23:00:11.566233 3088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 23:00:11.572299 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 3088
Aug 08 23:00:11 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230011, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM
Aug 08 23:00:11 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230011, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.60@KRBTEST.COM
Aug 08 23:00:11 dist-test-slave-f0tn krb5kdc[1363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230011, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.61@KRBTEST.COM
I20260808 23:00:11.578958 2398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.566868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:60233 (local address 127.23.194.61:38237)
0808 23:00:11.566911 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.566913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.567491 (+ 578us) server_negotiation.cc:407] Connection header received
0808 23:00:11.568258 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.568260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.568273 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.568319 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.569047 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.569842 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.571197 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.571464 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.571527 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.574719 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.574721 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.574721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.574729 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.576434 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.577918 (+ 1484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.577919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.577919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.577980 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.578248 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.578249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.578250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.578375 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.578430 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.578791 (+ 361us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.578866 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 23:00:11.579128 3308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.567160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:38237 (local address 127.23.194.3:60233)
0808 23:00:11.567420 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.567430 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.568170 (+ 740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.568405 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.568409 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.568769 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:11.568951 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.568956 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.569955 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.569956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.571079 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.571081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.571581 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.571582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.571667 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.572124 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:11.572128 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:11.574613 (+ 2485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:11.576521 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.576528 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.576530 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.576824 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.578047 (+ 1223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.578048 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.578048 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.578179 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.578456 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:11.578463 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:11.578627 (+ 164us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.578755 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.578916 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":67,"thread_start_us":198,"threads_started":1}
I20260808 23:00:11.579710 3298 heartbeater.cc:347] Connected to a master server at 127.23.194.61:38237
I20260808 23:00:11.579813 3298 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:11.580011 3298 heartbeater.cc:511] Master 127.23.194.61:38237 requested a full tablet report, sending...
I20260808 23:00:11.580430 2351 ts_manager.cc:194] Registered new tserver with Master: 5d4dcce6f4d54254a9b9a8c62290a0e4 (127.23.194.3:40999)
I20260808 23:00:11.581311 3303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.566884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:46459 (local address 127.23.194.3:40857)
0808 23:00:11.567118 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.567156 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.568163 (+ 1007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.568406 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.568410 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.568785 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:11.568952 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.568958 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.569946 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.569947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.570892 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.570894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.571276 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.571277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.571407 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.571864 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:11.571873 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:11.574264 (+ 2391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:11.577623 (+ 3359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.577624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.577624 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.577836 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.579537 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.579538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.579538 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.579597 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.581008 (+ 1411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:11.581009 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:11.581051 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.581110 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.581209 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":128,"thread_start_us":85,"threads_started":1}
I20260808 23:00:11.581507 3315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.573506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39016 (local address 127.23.194.60:46459)
0808 23:00:11.573811 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.573812 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.573819 (+ 7us) server_negotiation.cc:407] Connection header received
0808 23:00:11.573840 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.573841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.573853 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.573919 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.574730 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.575465 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.576628 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.576904 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.576956 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.578425 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.578426 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.578427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.578433 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.580249 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.580598 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.580599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.580599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.580657 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.580842 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.580843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.580843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.580938 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.581015 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.581315 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.581419 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":54,"threads_started":1}
I20260808 23:00:11.581657 2899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.567235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:40857 (local address 127.23.194.60:46459)
0808 23:00:11.567276 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.567278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.567470 (+ 192us) server_negotiation.cc:407] Connection header received
0808 23:00:11.568281 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.568283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.568301 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.568330 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.569071 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.569833 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.570976 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.571196 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.571259 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.574975 (+ 3716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.574977 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.574977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.574985 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.577536 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.579427 (+ 1891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.579428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.579428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.579486 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.580788 (+ 1302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.580789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.580789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.580924 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.580982 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.581503 (+ 521us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.581568 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 23:00:11.582314 3296 heartbeater.cc:347] Connected to a master server at 127.23.194.60:46459
I20260808 23:00:11.582401 3296 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:11.582531 3296 heartbeater.cc:511] Master 127.23.194.60:46459 requested a full tablet report, sending...
I20260808 23:00:11.582862 2624 ts_manager.cc:194] Registered new tserver with Master: 5d4dcce6f4d54254a9b9a8c62290a0e4 (127.23.194.3:40999)
I20260808 23:00:11.583568 3317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.573623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36084 (local address 127.23.194.61:38237)
0808 23:00:11.573739 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.573741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.574087 (+ 346us) server_negotiation.cc:407] Connection header received
0808 23:00:11.574111 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.574111 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.574124 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.574811 (+ 687us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.574846 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.575511 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.576384 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.576667 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.578345 (+ 1678us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.578394 (+ 49us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.578394 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.578395 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.578399 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.581186 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.582691 (+ 1505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.582692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.582693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.582756 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.582974 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.582975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.582976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.583148 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.583247 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.583398 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.583477 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":64,"threads_started":1}
I20260808 23:00:11.585034 3321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.573737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56920 (local address 127.23.194.62:35945)
0808 23:00:11.576558 (+ 2821us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.576560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.576566 (+ 6us) server_negotiation.cc:407] Connection header received
0808 23:00:11.576588 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.576588 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.576600 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.577321 (+ 721us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.577362 (+ 41us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.578116 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.579209 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.579501 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.579868 (+ 367us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.581045 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.581047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.581047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.581054 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.583836 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.584186 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.584187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.584187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.584263 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.584425 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.584425 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.584426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.584543 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.584623 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.584727 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.584815 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2788,"thread_start_us":90,"threads_started":1}
I20260808 23:00:11.585039 1513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.567180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:43649 (local address 127.23.194.62:35945)
0808 23:00:11.567274 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.567276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.567287 (+ 11us) server_negotiation.cc:407] Connection header received
0808 23:00:11.568282 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.568284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.568297 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.568329 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.569435 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.569956 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.571179 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.571427 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.571480 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.582071 (+ 10591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.582072 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.582072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.582080 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.583847 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.584172 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.584173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.584174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.584243 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.584439 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.584440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.584441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.584564 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.584615 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.584752 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.584877 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":85,"server-negotiator.queue_time_us":67}
I20260808 23:00:11.585085 3304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.566883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.23.194.3:43649)
0808 23:00:11.567110 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.567156 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.568154 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.568406 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.568410 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.568763 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:11.568952 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.568958 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.570034 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.570035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.571078 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.571081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.571499 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.571500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.571569 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.571981 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:11.571985 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:11.581968 (+ 9983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:11.583923 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.583924 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.583924 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.584084 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.584320 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:11.584320 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:11.584320 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:11.584380 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:11.584621 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:11.584622 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:11.584662 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.584722 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.584822 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":46,"spinlock_wait_cycles":7552,"thread_start_us":100,"threads_started":1}
I20260808 23:00:11.585602 3297 heartbeater.cc:347] Connected to a master server at 127.23.194.62:35945
I20260808 23:00:11.585665 3297 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:11.585731 3297 heartbeater.cc:511] Master 127.23.194.62:35945 requested a full tablet report, sending...
I20260808 23:00:11.585999 1430 ts_manager.cc:194] Re-registered known tserver with Master: 5d4dcce6f4d54254a9b9a8c62290a0e4 (127.23.194.3:40999)
I20260808 23:00:11.586419 1430 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:43649
I20260808 23:00:11.586717 1431 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:00:11.588573 1431 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:00:11.589320 2286 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: NON_VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: true } } }
I20260808 23:00:11.589437 2286 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:11.590653 2285 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5: attempting to promote NON_VOTER 8fa090f9309c484abd2d50c073281126 to VOTER
I20260808 23:00:11.590780 2285 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: false } }
I20260808 23:00:11.591153 2364 raft_consensus.cc:1261] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 1 FOLLOWER]: Refusing update from remote peer bc466d1f87bb4f4387cffdf35a21a5e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 23:00:11.591245 2626 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 1 LEARNER]: Refusing update from remote peer bc466d1f87bb4f4387cffdf35a21a5e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 23:00:11.591372 2286 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 23:00:11.591758 2286 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 23:00:11.592010 2287 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 8fa090f9309c484abd2d50c073281126 (127.23.194.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: false } } }
I20260808 23:00:11.592139 2364 raft_consensus.cc:2932] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 8fa090f9309c484abd2d50c073281126 (127.23.194.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: false } } }
I20260808 23:00:11.593026 2287 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: false } } }
I20260808 23:00:11.593104 2287 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:11.593076 2571 sys_catalog.cc:455] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: false } } }
I20260808 23:00:11.593163 2571 sys_catalog.cc:458] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:00:11.593156 2287 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: false } } }
I20260808 23:00:11.593195 2287 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:11.593832 2287 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: false } } }
I20260808 23:00:11.593914 2287 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:11.594172 2626 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 8fa090f9309c484abd2d50c073281126 (127.23.194.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: false } } }
I20260808 23:00:11.595296 2682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: false } } }
I20260808 23:00:11.595389 2682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:00:11.598279 3333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.594128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:40999 (local address 127.0.0.1:42238)
0808 23:00:11.594365 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.594377 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.594429 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.594833 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.594834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.594836 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:11.594883 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.594885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.596200 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.596201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.597060 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.597064 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.598038 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.598039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.598105 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.598110 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.598158 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.598207 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215}
I20260808 23:00:11.598387 3335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.594380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42238 (local address 127.23.194.3:40999)
0808 23:00:11.594652 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.594669 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.594680 (+ 11us) server_negotiation.cc:407] Connection header received
0808 23:00:11.594719 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.594721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.594738 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.594819 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:11.594943 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.596118 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.597135 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.597973 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.598038 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.598203 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.598233 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.598274 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":78,"threads_started":1}
I20260808 23:00:11.599057 3188 tablet_service.cc:1467] Processing CreateTablet for tablet d8404fbcec244f22aa58a201aa41f6a6 (DEFAULT_TABLE table=test-table [id=72fe38478c7e48c1b8bf6cf320a6c648]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:11.599426 3188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8404fbcec244f22aa58a201aa41f6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:11.599822 2284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.594208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:42459 (local address 127.0.0.1:34014)
0808 23:00:11.595336 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.595349 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.595399 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.595931 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.595932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.595934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:11.595972 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.595973 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.597344 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.597345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.598482 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.598485 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.599606 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.599608 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.599670 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.599674 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.599721 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.599760 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1109,"thread_start_us":102,"threads_started":1}
I20260808 23:00:11.599961 3336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.594629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34014 (local address 127.23.194.2:42459)
0808 23:00:11.594783 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.594787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.595796 (+ 1009us) server_negotiation.cc:407] Connection header received
0808 23:00:11.595843 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.595845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.595862 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.595934 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:11.596042 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.597248 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.598701 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.599534 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.599607 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.599770 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.599799 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.599838 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
I20260808 23:00:11.600026 3334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.594216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:36397 (local address 127.0.0.1:53028)
0808 23:00:11.596064 (+ 1848us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.596074 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.596129 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.596312 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.596313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.596315 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:11.596438 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.596440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.597642 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.597643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.598770 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.598772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.599802 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.599803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.599868 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.599873 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.599921 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.599962 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1791,"thread_start_us":91,"threads_started":1}
I20260808 23:00:11.600245 3338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.595497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53028 (local address 127.23.194.1:36397)
0808 23:00:11.595795 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.595800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.596148 (+ 348us) server_negotiation.cc:407] Connection header received
0808 23:00:11.596206 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.596208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.596225 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.596308 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:11.596510 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.597561 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.598850 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.599634 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.599689 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.599765 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.600122 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.600178 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":93,"threads_started":1}
I20260808 23:00:11.600911 2793 tablet_service.cc:1467] Processing CreateTablet for tablet d8404fbcec244f22aa58a201aa41f6a6 (DEFAULT_TABLE table=test-table [id=72fe38478c7e48c1b8bf6cf320a6c648]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:11.601274 2793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8404fbcec244f22aa58a201aa41f6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:11.602041 2993 tablet_service.cc:1467] Processing CreateTablet for tablet d8404fbcec244f22aa58a201aa41f6a6 (DEFAULT_TABLE table=test-table [id=72fe38478c7e48c1b8bf6cf320a6c648]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:11.602382 2993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8404fbcec244f22aa58a201aa41f6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:11.604105 3339 tablet_bootstrap.cc:492] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4: Bootstrap starting.
I20260808 23:00:11.605399 3339 tablet_bootstrap.cc:654] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:11.606117 3339 log.cc:824] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:11.606413 3340 tablet_bootstrap.cc:492] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c: Bootstrap starting.
I20260808 23:00:11.607201 3342 tablet_bootstrap.cc:492] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075: Bootstrap starting.
I20260808 23:00:11.607146 3339 tablet_bootstrap.cc:492] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4: No bootstrap required, opened a new log
I20260808 23:00:11.607417 3339 ts_tablet_manager.cc:1397] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260808 23:00:11.607550 3340 tablet_bootstrap.cc:654] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:11.608201 3342 tablet_bootstrap.cc:654] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:11.608557 3340 log.cc:824] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:11.608949 3342 log.cc:824] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:11.609521 3340 tablet_bootstrap.cc:492] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c: No bootstrap required, opened a new log
I20260808 23:00:11.609644 3340 ts_tablet_manager.cc:1397] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260808 23:00:11.609753 3342 tablet_bootstrap.cc:492] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075: No bootstrap required, opened a new log
I20260808 23:00:11.609807 3342 ts_tablet_manager.cc:1397] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 23:00:11.611097 3339 raft_consensus.cc:348] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 } } peers { permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 } } peers { permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 } }
I20260808 23:00:11.611291 3339 raft_consensus.cc:374] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:11.611332 3339 raft_consensus.cc:729] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4dcce6f4d54254a9b9a8c62290a0e4, State: Initialized, Role: FOLLOWER
I20260808 23:00:11.611479 3339 consensus_queue.cc:260] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 } } peers { permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 } } peers { permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 } }
I20260808 23:00:11.611811 3339 ts_tablet_manager.cc:1428] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 23:00:11.611961 3297 heartbeater.cc:503] Master 127.23.194.62:35945 was elected leader, sending a full tablet report...
I20260808 23:00:11.614038 3340 raft_consensus.cc:348] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 } } peers { permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 } } peers { permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 } }
I20260808 23:00:11.614202 3340 raft_consensus.cc:374] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:11.614226 3340 raft_consensus.cc:729] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 330d9fc4eb7b4a089b33ab13a463351c, State: Initialized, Role: FOLLOWER
I20260808 23:00:11.614274 3342 raft_consensus.cc:348] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 } } peers { permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 } } peers { permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 } }
I20260808 23:00:11.614334 3340 consensus_queue.cc:260] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 } } peers { permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 } } peers { permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 } }
I20260808 23:00:11.614403 3342 raft_consensus.cc:374] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:11.614436 3342 raft_consensus.cc:729] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b85349630b484dbd4ea782a8009075, State: Initialized, Role: FOLLOWER
I20260808 23:00:11.614538 3342 consensus_queue.cc:260] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 } } peers { permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 } } peers { permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 } }
I20260808 23:00:11.614622 3340 ts_tablet_manager.cc:1428] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260808 23:00:11.614749 2893 heartbeater.cc:503] Master 127.23.194.62:35945 was elected leader, sending a full tablet report...
I20260808 23:00:11.614827 3342 ts_tablet_manager.cc:1428] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260808 23:00:11.614939 3078 heartbeater.cc:503] Master 127.23.194.62:35945 was elected leader, sending a full tablet report...
I20260808 23:00:11.645175 3351 raft_consensus.cc:482] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:00:11.645287 3351 raft_consensus.cc:504] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 } } peers { permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 } } peers { permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 } }
I20260808 23:00:11.645679 3351 leader_election.cc:288] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d4dcce6f4d54254a9b9a8c62290a0e4 (127.23.194.3:40999), 330d9fc4eb7b4a089b33ab13a463351c (127.23.194.1:36397)
I20260808 23:00:11.649230 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.645928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:36397 (local address 127.23.194.2:39359)
0808 23:00:11.645973 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.645984 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.646026 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.646191 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.646192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.646193 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:11.646317 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.646319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.647140 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.647140 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.648413 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.648416 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.648937 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.648938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.649066 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.649070 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.649117 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.649158 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 23:00:11.649250 3338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.646027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:39359 (local address 127.23.194.1:36397)
0808 23:00:11.646090 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.646092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.646100 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:00:11.646122 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.646122 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.646130 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.646173 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:11.646376 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.647073 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.648491 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.648870 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.648925 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.648932 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.649140 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.649194 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 23:00:11.649407 3335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.645884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:42157 (local address 127.23.194.3:40999)
0808 23:00:11.645936 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.645938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.645947 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:00:11.646010 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.646011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.646021 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.646065 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:11.646329 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.647068 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.648532 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.648907 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.648979 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.648987 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.649278 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.649323 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 23:00:11.649472 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.645828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:40999 (local address 127.23.194.2:42157)
0808 23:00:11.645888 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:11.645898 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:11.645950 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:11.646091 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:11.646092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:11.646093 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:11.646260 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.646262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.647140 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.647140 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.648415 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:11.648416 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.649005 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.649006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:11.649210 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.649214 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:00:11.649271 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 23:00:11.649314 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":3}
I20260808 23:00:11.649809 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8404fbcec244f22aa58a201aa41f6a6" candidate_uuid: "c3b85349630b484dbd4ea782a8009075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "330d9fc4eb7b4a089b33ab13a463351c" is_pre_election: true
I20260808 23:00:11.649938 2820 raft_consensus.cc:2454] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3b85349630b484dbd4ea782a8009075 in term 0.
I20260808 23:00:11.650068 3213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8404fbcec244f22aa58a201aa41f6a6" candidate_uuid: "c3b85349630b484dbd4ea782a8009075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" is_pre_election: true
I20260808 23:00:11.650182 2929 leader_election.cc:302] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 330d9fc4eb7b4a089b33ab13a463351c, c3b85349630b484dbd4ea782a8009075; no voters:
I20260808 23:00:11.650238 3213 raft_consensus.cc:2454] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3b85349630b484dbd4ea782a8009075 in term 0.
I20260808 23:00:11.650302 3351 raft_consensus.cc:2781] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:00:11.650365 3351 raft_consensus.cc:482] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:00:11.650395 3351 raft_consensus.cc:3037] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:11.651473 3351 raft_consensus.cc:504] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 } } peers { permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 } } peers { permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 } }
I20260808 23:00:11.651582 3351 leader_election.cc:288] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [CANDIDATE]: Term 1 election: Requested vote from peers 5d4dcce6f4d54254a9b9a8c62290a0e4 (127.23.194.3:40999), 330d9fc4eb7b4a089b33ab13a463351c (127.23.194.1:36397)
I20260808 23:00:11.651760 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8404fbcec244f22aa58a201aa41f6a6" candidate_uuid: "c3b85349630b484dbd4ea782a8009075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "330d9fc4eb7b4a089b33ab13a463351c"
I20260808 23:00:11.651830 3213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8404fbcec244f22aa58a201aa41f6a6" candidate_uuid: "c3b85349630b484dbd4ea782a8009075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4"
I20260808 23:00:11.651852 2820 raft_consensus.cc:3037] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:11.651901 3213 raft_consensus.cc:3037] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:11.653213 2820 raft_consensus.cc:2454] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b85349630b484dbd4ea782a8009075 in term 1.
I20260808 23:00:11.653213 3213 raft_consensus.cc:2454] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b85349630b484dbd4ea782a8009075 in term 1.
I20260808 23:00:11.653414 2929 leader_election.cc:302] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 330d9fc4eb7b4a089b33ab13a463351c, c3b85349630b484dbd4ea782a8009075; no voters:
I20260808 23:00:11.653501 3351 raft_consensus.cc:2781] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:11.653676 3351 raft_consensus.cc:686] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [term 1 LEADER]: Becoming Leader. State: Replica: c3b85349630b484dbd4ea782a8009075, State: Running, Role: LEADER
I20260808 23:00:11.653770 3351 consensus_queue.cc:237] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 } } peers { permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 } } peers { permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 } }
I20260808 23:00:11.654559 1431 catalog_manager.cc:5495] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3b85349630b484dbd4ea782a8009075 (127.23.194.2). New cstate: current_term: 1 leader_uuid: "c3b85349630b484dbd4ea782a8009075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 } health_report { overall_health: UNKNOWN } } }
W20260808 23:00:11.665102 3080 tablet.cc:2369] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:00:11.678931 3336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.676708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34022 (local address 127.23.194.2:42459)
0808 23:00:11.676748 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.676750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.676830 (+ 80us) server_negotiation.cc:407] Connection header received
0808 23:00:11.676905 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.676906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.676923 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.676977 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:00:11.677087 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.677657 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.678387 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.678563 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.678603 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.678699 (+ 96us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:00:11.678796 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:00:11.678847 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.678859 (+ 12us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.678876 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 23:00:11.681253 3213 raft_consensus.cc:1261] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4 [term 1 FOLLOWER]: Refusing update from remote peer c3b85349630b484dbd4ea782a8009075: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:00:11.681324 2820 raft_consensus.cc:1261] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c [term 1 FOLLOWER]: Refusing update from remote peer c3b85349630b484dbd4ea782a8009075: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:00:11.681674 3351 consensus_queue.cc:1035] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:11.681902 3352 consensus_queue.cc:1035] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:11.684798 3359 mvcc.cc:205] Tried to move back new op lower bound from 7316398127843618816 to 7316398127734460416. Current Snapshot: MvccSnapshot[applied={T|T < 7316398127843618816}]
I20260808 23:00:11.685608 3362 mvcc.cc:205] Tried to move back new op lower bound from 7316398127843618816 to 7316398127734460416. Current Snapshot: MvccSnapshot[applied={T|T < 7316398127843618816}]
I20260808 23:00:11.685771 3360 mvcc.cc:205] Tried to move back new op lower bound from 7316398127843618816 to 7316398127734460416. Current Snapshot: MvccSnapshot[applied={T|T < 7316398127843618816 or (T in {7316398127843618816})}]
I20260808 23:00:11.688865 1431 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:00:11.688962 1431 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:00:11.690295 1431 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc466d1f87bb4f4387cffdf35a21a5e5: Sending DeleteTablet for 3 replicas of tablet d8404fbcec244f22aa58a201aa41f6a6
I20260808 23:00:11.690665 2993 tablet_service.cc:1514] Processing DeleteTablet for tablet d8404fbcec244f22aa58a201aa41f6a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:11 UTC) from {username='oryx', principal='oryx/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:34014
I20260808 23:00:11.691066 3188 tablet_service.cc:1514] Processing DeleteTablet for tablet d8404fbcec244f22aa58a201aa41f6a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:11 UTC) from {username='oryx', principal='oryx/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:42238
I20260808 23:00:11.691146 2793 tablet_service.cc:1514] Processing DeleteTablet for tablet d8404fbcec244f22aa58a201aa41f6a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:11 UTC) from {username='oryx', principal='oryx/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:53028
I20260808 23:00:11.691319 3372 tablet_replica.cc:330] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075: stopping tablet replica
I20260808 23:00:11.691425 3372 raft_consensus.cc:2229] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:00:11.691551 3372 raft_consensus.cc:2258] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:11.691787 3374 tablet_replica.cc:330] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c: stopping tablet replica
I20260808 23:00:11.691883 3374 raft_consensus.cc:2229] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:00:11.691980 3374 raft_consensus.cc:2258] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:11.692028 3372 ts_tablet_manager.cc:1905] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:11.692413 3374 ts_tablet_manager.cc:1905] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:11.693061 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 1380
I20260808 23:00:11.694648 3373 tablet_replica.cc:330] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4: stopping tablet replica
I20260808 23:00:11.694718 3372 ts_tablet_manager.cc:1918] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:00:11.694756 3373 raft_consensus.cc:2229] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:00:11.694775 3372 log.cc:1196] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/d8404fbcec244f22aa58a201aa41f6a6
I20260808 23:00:11.694854 3373 raft_consensus.cc:2258] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:11.695029 3372 ts_tablet_manager.cc:1939] T d8404fbcec244f22aa58a201aa41f6a6 P c3b85349630b484dbd4ea782a8009075: Deleting consensus metadata
I20260808 23:00:11.695025 3374 ts_tablet_manager.cc:1918] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:00:11.695083 3374 log.cc:1196] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-0/wal/wals/d8404fbcec244f22aa58a201aa41f6a6
I20260808 23:00:11.695317 3374 ts_tablet_manager.cc:1939] T d8404fbcec244f22aa58a201aa41f6a6 P 330d9fc4eb7b4a089b33ab13a463351c: Deleting consensus metadata
I20260808 23:00:11.699280 3373 ts_tablet_manager.cc:1905] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:11.700996 3373 ts_tablet_manager.cc:1918] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:00:11.701057 3373 log.cc:1196] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/d8404fbcec244f22aa58a201aa41f6a6
I20260808 23:00:11.701278 3373 ts_tablet_manager.cc:1939] T d8404fbcec244f22aa58a201aa41f6a6 P 5d4dcce6f4d54254a9b9a8c62290a0e4: Deleting consensus metadata
W20260808 23:00:11.712194 2726 connection.cc:535] server connection from 127.0.0.1:53028 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 23:00:11.712459 2926 connection.cc:535] server connection from 127.0.0.1:34014 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 23:00:11.712700 3123 connection.cc:535] server connection from 127.0.0.1:42238 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 23:00:11.713315 2899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.692221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39022 (local address 127.23.194.60:46459)
0808 23:00:11.692267 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.692269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.705711 (+ 13442us) server_negotiation.cc:407] Connection header received
0808 23:00:11.705747 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.705748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.705762 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.705839 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.706601 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.707404 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.708453 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.708776 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.708849 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.710413 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.710415 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.710415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.710424 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.712007 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.712404 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.712405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.712405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.712472 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.712668 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.712669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.712670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.712803 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.712902 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.713115 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.713216 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 23:00:11.721125 2899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.714571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39036 (local address 127.23.194.60:46459)
0808 23:00:11.714773 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.714774 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.714871 (+ 97us) server_negotiation.cc:407] Connection header received
0808 23:00:11.714966 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.714967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.714977 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.715004 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.715733 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.716464 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.717199 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.717406 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.717459 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.718759 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.718761 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.718761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.718768 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.720086 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.720372 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.720373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.720374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.720422 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.720590 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.720590 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.720591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.720713 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.720807 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.720955 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.721031 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181}
I20260808 23:00:11.722587 3317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:11.714683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36096 (local address 127.23.194.61:38237)
0808 23:00:11.714736 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:11.714737 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:11.714817 (+ 80us) server_negotiation.cc:407] Connection header received
0808 23:00:11.714917 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:11.714918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:11.714933 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:11.714963 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:11.716061 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.716536 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.717948 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:11.718271 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:11.718350 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:11.719833 (+ 1483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:11.719835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:11.719835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:11.719843 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:11.721536 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.721900 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.721901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.721902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.721957 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:11.722124 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:11.722124 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:11.722125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:11.722232 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:11.722314 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:11.722410 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 23:00:11.722490 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 23:00:12.687311 3389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:12.686966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.23.194.2:60427)
0808 23:00:12.687215 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:12.687276 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":72,"threads_started":1}
I20260808 23:00:12.687307 3388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:12.686967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.23.194.3:44981)
0808 23:00:12.687113 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:12.687241 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
W20260808 23:00:12.687525 3078 heartbeater.cc:644] Failed to heartbeat to 127.23.194.62:35945 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
W20260808 23:00:12.687572 3297 heartbeater.cc:644] Failed to heartbeat to 127.23.194.62:35945 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
I20260808 23:00:12.687736 3391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:12.687300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.23.194.1:57371)
0808 23:00:12.687604 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:12.687693 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":46,"threads_started":1}
W20260808 23:00:12.687912 2893 heartbeater.cc:644] Failed to heartbeat to 127.23.194.62:35945 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
I20260808 23:00:13.191273 3392 raft_consensus.cc:482] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc466d1f87bb4f4387cffdf35a21a5e5)
I20260808 23:00:13.191285 3395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc466d1f87bb4f4387cffdf35a21a5e5)
I20260808 23:00:13.191411 3392 raft_consensus.cc:504] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: false } }
I20260808 23:00:13.191425 3395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: false } }
I20260808 23:00:13.191684 3392 leader_election.cc:288] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bc466d1f87bb4f4387cffdf35a21a5e5 (127.23.194.62:35945), 8fa090f9309c484abd2d50c073281126 (127.23.194.60:46459)
I20260808 23:00:13.191916 3395 leader_election.cc:288] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bc466d1f87bb4f4387cffdf35a21a5e5 (127.23.194.62:35945), faec855a53e74875a88bbb0c1a5107a6 (127.23.194.61:38237)
I20260808 23:00:13.192042 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.191812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.0.0.1:57468)
0808 23:00:13.191930 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.192002 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"threads_started":1}
I20260808 23:00:13.192163 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.191816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.0.0.1:57460)
0808 23:00:13.192075 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.192123 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":54,"threads_started":1}
W20260808 23:00:13.192283 2605 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
W20260808 23:00:13.192323 2329 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
I20260808 23:00:13.192761 3403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.192649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.0.0.1:57480)
0808 23:00:13.192699 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.192733 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 23:00:13.192814 3409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.192722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.0.0.1:57486)
0808 23:00:13.192760 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.192786 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 23:00:13.192836 2329 leader_election.cc:334] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bc466d1f87bb4f4387cffdf35a21a5e5 (127.23.194.62:35945): Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
W20260808 23:00:13.192886 2605 leader_election.cc:334] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bc466d1f87bb4f4387cffdf35a21a5e5 (127.23.194.62:35945): Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
I20260808 23:00:13.195307 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.192030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:46459 (local address 127.0.0.1:39038)
0808 23:00:13.192215 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.192223 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:13.192276 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:13.192596 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:13.192597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:13.192600 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:13.192666 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:13.192668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.193592 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.193593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:13.194320 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:13.194323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.195037 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.195038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:13.195105 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:13.195110 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:00:13.195177 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 23:00:13.195218 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":1,"thread_start_us":52,"threads_started":1}
I20260808 23:00:13.195302 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.191840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:38237 (local address 127.0.0.1:36112)
0808 23:00:13.192090 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.192099 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:13.192216 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:13.192428 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:13.192429 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:13.192431 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:13.192492 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:13.192494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.193294 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.193294 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:13.194373 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:13.194376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.195073 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.195073 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:13.195133 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:13.195137 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:00:13.195185 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 23:00:13.195225 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":46,"threads_started":1}
I20260808 23:00:13.195318 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.191906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36112 (local address 127.23.194.61:38237)
0808 23:00:13.192030 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:13.192032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:13.192184 (+ 152us) server_negotiation.cc:407] Connection header received
0808 23:00:13.192294 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:13.192296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:13.192306 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:13.192352 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:13.192571 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.193216 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.194483 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.194995 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.195049 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:13.195058 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:13.195193 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 23:00:13.195234 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":54,"threads_started":1}
I20260808 23:00:13.195684 2364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8fa090f9309c484abd2d50c073281126" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "faec855a53e74875a88bbb0c1a5107a6" is_pre_election: true
I20260808 23:00:13.195782 2364 raft_consensus.cc:2454] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fa090f9309c484abd2d50c073281126 in term 1.
I20260808 23:00:13.195304 3408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.192132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39038 (local address 127.23.194.60:46459)
0808 23:00:13.192458 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:13.192460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:13.192468 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:00:13.192491 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:13.192491 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:13.192501 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:13.192728 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:13.192761 (+ 33us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.193524 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.194398 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.194957 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.195028 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:13.195042 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:13.195179 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 23:00:13.195226 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":46,"threads_started":1}
I20260808 23:00:13.195935 2602 leader_election.cc:302] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fa090f9309c484abd2d50c073281126, faec855a53e74875a88bbb0c1a5107a6; no voters: bc466d1f87bb4f4387cffdf35a21a5e5
I20260808 23:00:13.195943 2626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "faec855a53e74875a88bbb0c1a5107a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8fa090f9309c484abd2d50c073281126" is_pre_election: true
I20260808 23:00:13.196053 3395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 23:00:13.196097 3395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc466d1f87bb4f4387cffdf35a21a5e5)
I20260808 23:00:13.196123 3395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 1 FOLLOWER]: Advancing to term 2
I20260808 23:00:13.196774 3395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: false } }
I20260808 23:00:13.196892 2626 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate faec855a53e74875a88bbb0c1a5107a6 in current term 2: Already voted for candidate 8fa090f9309c484abd2d50c073281126 in this term.
I20260808 23:00:13.196893 3395 leader_election.cc:288] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [CANDIDATE]: Term 2 election: Requested vote from peers bc466d1f87bb4f4387cffdf35a21a5e5 (127.23.194.62:35945), faec855a53e74875a88bbb0c1a5107a6 (127.23.194.61:38237)
I20260808 23:00:13.197036 2364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8fa090f9309c484abd2d50c073281126" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "faec855a53e74875a88bbb0c1a5107a6"
I20260808 23:00:13.197118 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.196959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.0.0.1:57498)
0808 23:00:13.197027 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.197054 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 23:00:13.197085 2328 leader_election.cc:302] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: faec855a53e74875a88bbb0c1a5107a6; no voters: 8fa090f9309c484abd2d50c073281126, bc466d1f87bb4f4387cffdf35a21a5e5
I20260808 23:00:13.197152 2364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 1 FOLLOWER]: Advancing to term 2
I20260808 23:00:13.197415 3409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.197324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.0.0.1:57500)
0808 23:00:13.197354 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.197393 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260808 23:00:13.197508 2605 leader_election.cc:334] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bc466d1f87bb4f4387cffdf35a21a5e5 (127.23.194.62:35945): Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
I20260808 23:00:13.197937 2364 raft_consensus.cc:2454] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fa090f9309c484abd2d50c073281126 in term 2.
I20260808 23:00:13.198043 3392 raft_consensus.cc:2726] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 23:00:13.198112 2602 leader_election.cc:302] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fa090f9309c484abd2d50c073281126, faec855a53e74875a88bbb0c1a5107a6; no voters: bc466d1f87bb4f4387cffdf35a21a5e5
I20260808 23:00:13.199681 3395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 23:00:13.199793 3395 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [term 2 LEADER]: Becoming Leader. State: Replica: 8fa090f9309c484abd2d50c073281126, State: Running, Role: LEADER
I20260808 23:00:13.199859 3395 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: false } }
I20260808 23:00:13.200163 3413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fa090f9309c484abd2d50c073281126. Latest consensus state: current_term: 2 leader_uuid: "8fa090f9309c484abd2d50c073281126" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: false } } }
I20260808 23:00:13.200237 3413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:13.200402 3417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:00:13.200881 3417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:00:13.200963 3417 catalog_manager.cc:1259] Loaded cluster ID: ad6ba00deee14b4ea2be0cd0685da656
I20260808 23:00:13.200994 3417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:00:13.201139 3417 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:00:13.201231 3417 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: Loaded TSK: 0
I20260808 23:00:13.201366 3417 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:00:13.255173 2622 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39036:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:00:13.256732 2622 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:00:13.257725 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.257625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.0.0.1:57522)
0808 23:00:13.257669 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.257698 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 23:00:13.257750 2364 raft_consensus.cc:1261] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [term 2 FOLLOWER]: Refusing update from remote peer 8fa090f9309c484abd2d50c073281126: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 23:00:13.258088 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.257963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.0.0.1:57526)
0808 23:00:13.258009 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.258039 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 23:00:13.258038 3413 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260808 23:00:13.258165 2605 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 -> Peer bc466d1f87bb4f4387cffdf35a21a5e5 (127.23.194.62:35945): Couldn't send request to peer bc466d1f87bb4f4387cffdf35a21a5e5. Status: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 23:00:13.258627 3392 sys_catalog.cc:455] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fa090f9309c484abd2d50c073281126. Latest consensus state: current_term: 2 leader_uuid: "8fa090f9309c484abd2d50c073281126" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: false } } }
I20260808 23:00:13.258721 3392 sys_catalog.cc:458] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:00:13.259023 3418 mvcc.cc:205] Tried to move back new op lower bound from 7316398134301782016 to 7316398134067027968. Current Snapshot: MvccSnapshot[applied={T|T < 7316398134301782016}]
I20260808 23:00:13.259121 3395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "8fa090f9309c484abd2d50c073281126" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: false } } }
I20260808 23:00:13.259213 3395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:13.259317 3392 sys_catalog.cc:455] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "8fa090f9309c484abd2d50c073281126" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc466d1f87bb4f4387cffdf35a21a5e5" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35945 } } peers { permanent_uuid: "faec855a53e74875a88bbb0c1a5107a6" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38237 } attrs { promote: false } } peers { permanent_uuid: "8fa090f9309c484abd2d50c073281126" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 46459 } attrs { promote: false } } }
I20260808 23:00:13.259408 3392 sys_catalog.cc:458] T 00000000000000000000000000000000 P faec855a53e74875a88bbb0c1a5107a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:00:13.264151 3431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.261205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53038 (local address 127.23.194.1:36397)
0808 23:00:13.261450 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:13.261452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:13.261460 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:00:13.261483 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:13.261484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:13.261493 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:13.261696 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:13.261752 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.262467 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.263376 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.263848 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.263908 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:13.263921 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:13.264042 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 23:00:13.264076 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":58,"threads_started":1}
I20260808 23:00:13.264148 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.261014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:36397 (local address 127.0.0.1:53038)
0808 23:00:13.261295 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.261304 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:13.261337 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:13.261574 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:13.261575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:13.261577 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:13.261607 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:13.261608 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.262560 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.262561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:13.263244 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:13.263246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.263919 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.263920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:13.263982 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:13.263987 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:00:13.264031 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 23:00:13.264072 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":3,"thread_start_us":83,"threads_started":1}
I20260808 23:00:13.264317 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.260980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:42459 (local address 127.0.0.1:34036)
0808 23:00:13.261156 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.261165 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:13.261208 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:13.261579 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:13.261580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:13.261582 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:13.261637 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:13.261639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.262276 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.262277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:13.263345 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:13.263347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.264083 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.264084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:13.264144 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:13.264148 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:00:13.264189 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 23:00:13.264229 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":2}
I20260808 23:00:13.264418 3409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.260930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:40999 (local address 127.0.0.1:42244)
0808 23:00:13.261016 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.261026 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:13.261077 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:13.261480 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:13.261481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:13.261483 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:13.261542 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:13.261544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.262513 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.262514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:13.263540 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:13.263542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.264239 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.264240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:13.264299 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:13.264302 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:00:13.264329 (+ 27us) client_negotiation.cc:240] Negotiation successful
0808 23:00:13.264363 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 23:00:13.264573 3427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.261044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34036 (local address 127.23.194.2:42459)
0808 23:00:13.261443 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:13.261445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:13.261453 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:00:13.261479 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:13.261480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:13.261490 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:13.261702 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:13.261748 (+ 46us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.262205 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.263439 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.263994 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.264426 (+ 432us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:13.264444 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:13.264463 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 23:00:13.264507 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":62,"threads_started":1}
I20260808 23:00:13.264642 3429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.261209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42244 (local address 127.23.194.3:40999)
0808 23:00:13.261324 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:13.261325 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:13.261334 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:00:13.261360 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:13.261361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:13.261371 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:13.261606 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:13.261654 (+ 48us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.262435 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.263623 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.264155 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.264507 (+ 352us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:13.264520 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:13.264538 (+ 18us) server_negotiation.cc:299] Negotiation successful
0808 23:00:13.264574 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":49,"threads_started":1}
I20260808 23:00:13.264706 2793 tablet_service.cc:1467] Processing CreateTablet for tablet c99edd8da4be4365b23df29d2ba2dc4e (DEFAULT_TABLE table=test-table [id=7e66328682e74d85b48be0660a59d144]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:13.264842 2793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c99edd8da4be4365b23df29d2ba2dc4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:13.265651 3188 tablet_service.cc:1467] Processing CreateTablet for tablet c99edd8da4be4365b23df29d2ba2dc4e (DEFAULT_TABLE table=test-table [id=7e66328682e74d85b48be0660a59d144]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:13.265697 2993 tablet_service.cc:1467] Processing CreateTablet for tablet c99edd8da4be4365b23df29d2ba2dc4e (DEFAULT_TABLE table=test-table [id=7e66328682e74d85b48be0660a59d144]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:13.265775 3188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c99edd8da4be4365b23df29d2ba2dc4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:13.265830 2993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c99edd8da4be4365b23df29d2ba2dc4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:13.266165 3433 tablet_bootstrap.cc:492] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c: Bootstrap starting.
I20260808 23:00:13.266610 3433 tablet_bootstrap.cc:654] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:13.267207 3433 tablet_bootstrap.cc:492] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c: No bootstrap required, opened a new log
I20260808 23:00:13.267230 3438 tablet_bootstrap.cc:492] T c99edd8da4be4365b23df29d2ba2dc4e P c3b85349630b484dbd4ea782a8009075: Bootstrap starting.
I20260808 23:00:13.267261 3433 ts_tablet_manager.cc:1397] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 23:00:13.267365 3433 raft_consensus.cc:348] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 } } peers { permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 } } peers { permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 } }
I20260808 23:00:13.267432 3433 raft_consensus.cc:374] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:13.267445 3433 raft_consensus.cc:729] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 330d9fc4eb7b4a089b33ab13a463351c, State: Initialized, Role: FOLLOWER
I20260808 23:00:13.267498 3433 consensus_queue.cc:260] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 } } peers { permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 } } peers { permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 } }
I20260808 23:00:13.267622 3438 tablet_bootstrap.cc:654] T c99edd8da4be4365b23df29d2ba2dc4e P c3b85349630b484dbd4ea782a8009075: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:13.267647 3433 ts_tablet_manager.cc:1428] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:13.267021 3437 tablet_bootstrap.cc:492] T c99edd8da4be4365b23df29d2ba2dc4e P 5d4dcce6f4d54254a9b9a8c62290a0e4: Bootstrap starting.
I20260808 23:00:13.268052 3441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.267860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.23.194.1:40211)
0808 23:00:13.267982 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.268019 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":78,"threads_started":1}
I20260808 23:00:13.268220 2891 heartbeater.cc:503] Master 127.23.194.60:46459 was elected leader, sending a full tablet report...
I20260808 23:00:13.268347 3437 tablet_bootstrap.cc:654] T c99edd8da4be4365b23df29d2ba2dc4e P 5d4dcce6f4d54254a9b9a8c62290a0e4: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:13.268886 3437 tablet_bootstrap.cc:492] T c99edd8da4be4365b23df29d2ba2dc4e P 5d4dcce6f4d54254a9b9a8c62290a0e4: No bootstrap required, opened a new log
I20260808 23:00:13.268910 3438 tablet_bootstrap.cc:492] T c99edd8da4be4365b23df29d2ba2dc4e P c3b85349630b484dbd4ea782a8009075: No bootstrap required, opened a new log
I20260808 23:00:13.268932 3437 ts_tablet_manager.cc:1397] T c99edd8da4be4365b23df29d2ba2dc4e P 5d4dcce6f4d54254a9b9a8c62290a0e4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:00:13.268949 3438 ts_tablet_manager.cc:1397] T c99edd8da4be4365b23df29d2ba2dc4e P c3b85349630b484dbd4ea782a8009075: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:00:13.269045 3438 raft_consensus.cc:348] T c99edd8da4be4365b23df29d2ba2dc4e P c3b85349630b484dbd4ea782a8009075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 } } peers { permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 } } peers { permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 } }
I20260808 23:00:13.269052 3437 raft_consensus.cc:348] T c99edd8da4be4365b23df29d2ba2dc4e P 5d4dcce6f4d54254a9b9a8c62290a0e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 } } peers { permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 } } peers { permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 } }
I20260808 23:00:13.269106 3438 raft_consensus.cc:374] T c99edd8da4be4365b23df29d2ba2dc4e P c3b85349630b484dbd4ea782a8009075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:13.269106 3437 raft_consensus.cc:374] T c99edd8da4be4365b23df29d2ba2dc4e P 5d4dcce6f4d54254a9b9a8c62290a0e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:13.269122 3438 raft_consensus.cc:729] T c99edd8da4be4365b23df29d2ba2dc4e P c3b85349630b484dbd4ea782a8009075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b85349630b484dbd4ea782a8009075, State: Initialized, Role: FOLLOWER
I20260808 23:00:13.269126 3437 raft_consensus.cc:729] T c99edd8da4be4365b23df29d2ba2dc4e P 5d4dcce6f4d54254a9b9a8c62290a0e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4dcce6f4d54254a9b9a8c62290a0e4, State: Initialized, Role: FOLLOWER
I20260808 23:00:13.269177 3437 consensus_queue.cc:260] T c99edd8da4be4365b23df29d2ba2dc4e P 5d4dcce6f4d54254a9b9a8c62290a0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 } } peers { permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 } } peers { permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 } }
I20260808 23:00:13.269186 3438 consensus_queue.cc:260] T c99edd8da4be4365b23df29d2ba2dc4e P c3b85349630b484dbd4ea782a8009075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 } } peers { permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 } } peers { permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 } }
I20260808 23:00:13.269326 3438 ts_tablet_manager.cc:1428] T c99edd8da4be4365b23df29d2ba2dc4e P c3b85349630b484dbd4ea782a8009075: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:13.269348 3437 ts_tablet_manager.cc:1428] T c99edd8da4be4365b23df29d2ba2dc4e P 5d4dcce6f4d54254a9b9a8c62290a0e4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:13.269758 3296 heartbeater.cc:503] Master 127.23.194.60:46459 was elected leader, sending a full tablet report...
I20260808 23:00:13.269960 3448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.269571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.23.194.2:54279)
0808 23:00:13.269887 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.269926 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":76,"threads_started":1}
I20260808 23:00:13.270170 3077 heartbeater.cc:503] Master 127.23.194.60:46459 was elected leader, sending a full tablet report...
I20260808 23:00:13.270496 3449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.269752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.23.194.3:58521)
0808 23:00:13.270427 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.270463 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":69,"threads_started":1}
I20260808 23:00:13.270787 3449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.270706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.23.194.3:51075)
0808 23:00:13.270743 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.270765 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 23:00:13.274849 3439 raft_consensus.cc:482] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:00:13.274904 3439 raft_consensus.cc:504] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 } } peers { permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 } } peers { permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 } }
I20260808 23:00:13.275146 3439 leader_election.cc:288] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d4dcce6f4d54254a9b9a8c62290a0e4 (127.23.194.3:40999), c3b85349630b484dbd4ea782a8009075 (127.23.194.2:42459)
I20260808 23:00:13.278119 3429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.275308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:41467 (local address 127.23.194.3:40999)
0808 23:00:13.275352 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:13.275354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:13.275363 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:00:13.275485 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:13.275486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:13.275494 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:13.275541 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:13.275678 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.276191 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.277296 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.277829 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.277906 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:13.277916 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:13.278051 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 23:00:13.278078 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 23:00:13.278144 3441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.275265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:40999 (local address 127.23.194.1:41467)
0808 23:00:13.275314 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.275324 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:13.275374 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:13.275559 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:13.275560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:13.275562 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:13.275607 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:13.275609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.276276 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.276277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:13.277179 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:13.277182 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.277905 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.277906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:13.277974 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:13.277979 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:00:13.278031 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 23:00:13.278065 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":41}
I20260808 23:00:13.278419 3450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.275278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:42459 (local address 127.23.194.1:33935)
0808 23:00:13.275432 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.275442 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:13.275485 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:13.275607 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:13.275608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:13.275609 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:13.275666 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:13.275668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.276235 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.276236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:13.277145 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:13.277148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.278226 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.278227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:13.278294 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:13.278298 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:00:13.278340 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 23:00:13.278365 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":23,"thread_start_us":97,"threads_started":1}
I20260808 23:00:13.278474 3427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.275450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:33935 (local address 127.23.194.2:42459)
0808 23:00:13.275502 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:13.275504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:13.275512 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:00:13.275536 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:13.275537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:13.275546 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:13.275604 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:13.275730 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.276162 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.277238 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.277818 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.277906 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:13.277916 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:13.278369 (+ 453us) server_negotiation.cc:299] Negotiation successful
0808 23:00:13.278417 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 23:00:13.278441 3213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99edd8da4be4365b23df29d2ba2dc4e" candidate_uuid: "330d9fc4eb7b4a089b33ab13a463351c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" is_pre_election: true
I20260808 23:00:13.278541 3213 raft_consensus.cc:2454] T c99edd8da4be4365b23df29d2ba2dc4e P 5d4dcce6f4d54254a9b9a8c62290a0e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 330d9fc4eb7b4a089b33ab13a463351c in term 0.
I20260808 23:00:13.278733 2727 leader_election.cc:302] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 330d9fc4eb7b4a089b33ab13a463351c, 5d4dcce6f4d54254a9b9a8c62290a0e4; no voters:
I20260808 23:00:13.278703 3022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99edd8da4be4365b23df29d2ba2dc4e" candidate_uuid: "330d9fc4eb7b4a089b33ab13a463351c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b85349630b484dbd4ea782a8009075" is_pre_election: true
I20260808 23:00:13.278822 3022 raft_consensus.cc:2454] T c99edd8da4be4365b23df29d2ba2dc4e P c3b85349630b484dbd4ea782a8009075 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 330d9fc4eb7b4a089b33ab13a463351c in term 0.
I20260808 23:00:13.278854 3439 raft_consensus.cc:2781] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:00:13.278891 3439 raft_consensus.cc:482] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:00:13.278920 3439 raft_consensus.cc:3037] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:13.279608 3439 raft_consensus.cc:504] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 } } peers { permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 } } peers { permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 } }
I20260808 23:00:13.279737 3439 leader_election.cc:288] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c [CANDIDATE]: Term 1 election: Requested vote from peers 5d4dcce6f4d54254a9b9a8c62290a0e4 (127.23.194.3:40999), c3b85349630b484dbd4ea782a8009075 (127.23.194.2:42459)
I20260808 23:00:13.279848 3022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99edd8da4be4365b23df29d2ba2dc4e" candidate_uuid: "330d9fc4eb7b4a089b33ab13a463351c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b85349630b484dbd4ea782a8009075"
W20260808 23:00:13.279891 2895 tablet.cc:2369] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:00:13.279918 3022 raft_consensus.cc:3037] T c99edd8da4be4365b23df29d2ba2dc4e P c3b85349630b484dbd4ea782a8009075 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:13.279910 3213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99edd8da4be4365b23df29d2ba2dc4e" candidate_uuid: "330d9fc4eb7b4a089b33ab13a463351c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4"
I20260808 23:00:13.279976 3213 raft_consensus.cc:3037] T c99edd8da4be4365b23df29d2ba2dc4e P 5d4dcce6f4d54254a9b9a8c62290a0e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:13.280545 3022 raft_consensus.cc:2454] T c99edd8da4be4365b23df29d2ba2dc4e P c3b85349630b484dbd4ea782a8009075 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 330d9fc4eb7b4a089b33ab13a463351c in term 1.
I20260808 23:00:13.280550 3213 raft_consensus.cc:2454] T c99edd8da4be4365b23df29d2ba2dc4e P 5d4dcce6f4d54254a9b9a8c62290a0e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 330d9fc4eb7b4a089b33ab13a463351c in term 1.
I20260808 23:00:13.280709 2728 leader_election.cc:302] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 330d9fc4eb7b4a089b33ab13a463351c, c3b85349630b484dbd4ea782a8009075; no voters:
I20260808 23:00:13.280807 3439 raft_consensus.cc:2781] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:13.280946 3439 raft_consensus.cc:686] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c [term 1 LEADER]: Becoming Leader. State: Replica: 330d9fc4eb7b4a089b33ab13a463351c, State: Running, Role: LEADER
I20260808 23:00:13.281008 3439 consensus_queue.cc:237] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 } } peers { permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 } } peers { permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 } }
I20260808 23:00:13.281229 3450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.281131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.23.194.1:48301)
0808 23:00:13.281172 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.281202 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 23:00:13.281769 2624 catalog_manager.cc:5495] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c reported cstate change: term changed from 0 to 1, leader changed from <none> to 330d9fc4eb7b4a089b33ab13a463351c (127.23.194.1). New cstate: current_term: 1 leader_uuid: "330d9fc4eb7b4a089b33ab13a463351c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "330d9fc4eb7b4a089b33ab13a463351c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36397 } health_report { overall_health: HEALTHY } } }
I20260808 23:00:13.292778 3431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.290804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53052 (local address 127.23.194.1:36397)
0808 23:00:13.290854 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:13.290855 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:13.290878 (+ 23us) server_negotiation.cc:407] Connection header received
0808 23:00:13.290902 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:13.290903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:13.290918 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:13.290964 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:00:13.291095 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.291519 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.292201 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:13.292365 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:13.292406 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:13.292535 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:00:13.292626 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:00:13.292688 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:13.292718 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 23:00:13.292733 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 23:00:13.293705 3213 raft_consensus.cc:1261] T c99edd8da4be4365b23df29d2ba2dc4e P 5d4dcce6f4d54254a9b9a8c62290a0e4 [term 1 FOLLOWER]: Refusing update from remote peer 330d9fc4eb7b4a089b33ab13a463351c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:00:13.293771 3022 raft_consensus.cc:1261] T c99edd8da4be4365b23df29d2ba2dc4e P c3b85349630b484dbd4ea782a8009075 [term 1 FOLLOWER]: Refusing update from remote peer 330d9fc4eb7b4a089b33ab13a463351c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:00:13.293964 3439 consensus_queue.cc:1035] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4dcce6f4d54254a9b9a8c62290a0e4" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 40999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:13.294108 3451 consensus_queue.cc:1035] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b85349630b484dbd4ea782a8009075" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 42459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:13.294711 3449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.294594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.23.194.3:60087)
0808 23:00:13.294641 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.294679 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 23:00:13.296100 3448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.295993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.23.194.2:47407)
0808 23:00:13.296048 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.296074 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 23:00:13.296125 3450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.296003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.23.194.1:44179)
0808 23:00:13.296069 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.296096 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 23:00:13.296471 3465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.296266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.23.194.2:38629)
0808 23:00:13.296390 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.296431 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":78,"threads_started":1}
I20260808 23:00:13.296548 3449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.296451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.23.194.3:44077)
0808 23:00:13.296497 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.296523 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 23:00:13.296634 3450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.296549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:35945 (local address 127.23.194.1:59023)
0808 23:00:13.296588 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.296611 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:35945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 23:00:13.298784 2622 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:00:13.298874 2622 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:00:13.300077 2622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8fa090f9309c484abd2d50c073281126: Sending DeleteTablet for 3 replicas of tablet c99edd8da4be4365b23df29d2ba2dc4e
I20260808 23:00:13.300423 2793 tablet_service.cc:1514] Processing DeleteTablet for tablet c99edd8da4be4365b23df29d2ba2dc4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:13 UTC) from {username='oryx', principal='oryx/127.23.194.60@KRBTEST.COM'} at 127.0.0.1:53038
I20260808 23:00:13.300429 3188 tablet_service.cc:1514] Processing DeleteTablet for tablet c99edd8da4be4365b23df29d2ba2dc4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:13 UTC) from {username='oryx', principal='oryx/127.23.194.60@KRBTEST.COM'} at 127.0.0.1:42244
I20260808 23:00:13.300513 2993 tablet_service.cc:1514] Processing DeleteTablet for tablet c99edd8da4be4365b23df29d2ba2dc4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:13 UTC) from {username='oryx', principal='oryx/127.23.194.60@KRBTEST.COM'} at 127.0.0.1:34036
I20260808 23:00:13.300755 3469 tablet_replica.cc:330] T c99edd8da4be4365b23df29d2ba2dc4e P 330d9fc4eb7b4a089b33ab13a463351c: stopping tablet replica
I20260808 23:00:13.300869 3472 tablet_replica.cc:330] T c99edd8da4be4365b23df29d2ba2dc4e P c3b85349630b484dbd4ea782a8009075: stopping tablet replica
I20260808 23:00:13.300927 3472 raft_consensus.cc:2229] T c99edd8da4be4365b23df29d2ba2dc4e P c3b85349630b484dbd4ea782a8009075 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:00:13.300980 3472 raft_consensus.cc:2258] T c99edd8da4be4365b23df29d2ba2dc4e P c3b85349630b484dbd4ea782a8009075 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:13.301019 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 2569
I20260808 23:00:13.303476 3467 tablet_replica.cc:330] T c99edd8da4be4365b23df29d2ba2dc4e P 5d4dcce6f4d54254a9b9a8c62290a0e4: stopping tablet replica
I20260808 23:00:13.303529 3467 raft_consensus.cc:2229] T c99edd8da4be4365b23df29d2ba2dc4e P 5d4dcce6f4d54254a9b9a8c62290a0e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:00:13.303563 3467 raft_consensus.cc:2258] T c99edd8da4be4365b23df29d2ba2dc4e P 5d4dcce6f4d54254a9b9a8c62290a0e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:13.303975 3472 ts_tablet_manager.cc:1905] T c99edd8da4be4365b23df29d2ba2dc4e P c3b85349630b484dbd4ea782a8009075: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:13.305896 3472 ts_tablet_manager.cc:1918] T c99edd8da4be4365b23df29d2ba2dc4e P c3b85349630b484dbd4ea782a8009075: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:00:13.305959 3472 log.cc:1196] T c99edd8da4be4365b23df29d2ba2dc4e P c3b85349630b484dbd4ea782a8009075: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/c99edd8da4be4365b23df29d2ba2dc4e
I20260808 23:00:13.306241 3472 ts_tablet_manager.cc:1939] T c99edd8da4be4365b23df29d2ba2dc4e P c3b85349630b484dbd4ea782a8009075: Deleting consensus metadata
I20260808 23:00:13.307444 3467 ts_tablet_manager.cc:1905] T c99edd8da4be4365b23df29d2ba2dc4e P 5d4dcce6f4d54254a9b9a8c62290a0e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:13.307693 2605 catalog_manager.cc:4841] TS c3b85349630b484dbd4ea782a8009075 (127.23.194.2:42459): tablet c99edd8da4be4365b23df29d2ba2dc4e (table test-table [id=7e66328682e74d85b48be0660a59d144]) successfully deleted
I20260808 23:00:13.308666 3467 ts_tablet_manager.cc:1918] T c99edd8da4be4365b23df29d2ba2dc4e P 5d4dcce6f4d54254a9b9a8c62290a0e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:00:13.308709 3467 log.cc:1196] T c99edd8da4be4365b23df29d2ba2dc4e P 5d4dcce6f4d54254a9b9a8c62290a0e4: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/c99edd8da4be4365b23df29d2ba2dc4e
I20260808 23:00:13.308890 3467 ts_tablet_manager.cc:1939] T c99edd8da4be4365b23df29d2ba2dc4e P 5d4dcce6f4d54254a9b9a8c62290a0e4: Deleting consensus metadata
I20260808 23:00:13.313771 2604 catalog_manager.cc:4841] TS 5d4dcce6f4d54254a9b9a8c62290a0e4 (127.23.194.3:40999): tablet c99edd8da4be4365b23df29d2ba2dc4e (table test-table [id=7e66328682e74d85b48be0660a59d144]) successfully deleted
I20260808 23:00:13.318089 3409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:13.317963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:36397 (local address 127.0.0.1:53058)
0808 23:00:13.318019 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:13.318053 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.1:36397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 23:00:13.318279 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 2901
W20260808 23:00:13.318264 2604 catalog_manager.cc:4584] TS 330d9fc4eb7b4a089b33ab13a463351c (127.23.194.1:36397): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c99edd8da4be4365b23df29d2ba2dc4e: Network error: Client connection negotiation failed: client connection to 127.23.194.1:36397: connect: Connection refused (error 111)
I20260808 23:00:13.332815 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 3088
I20260808 23:00:13.343937 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 2133
I20260808 23:00:13.351738 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 2424
2026-08-08T23:00:13Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4586 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:00:13 dist-test-slave-f0tn krb5kdc[3475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:00:13 dist-test-slave-f0tn krb5kdc[3475](info): set up 2 sockets
Aug 08 23:00:13 dist-test-slave-f0tn krb5kdc[3475](info): commencing operation
krb5kdc: starting...
W20260808 23:00:14.493161 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:00:14 dist-test-slave-f0tn krb5kdc[3475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:00:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:00:14Z Disabled control of system clock
WARNING: no policy specified for oryx/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.62.keytab.
Entry for principal oryx/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.62.keytab.
I20260808 23:00:14.612114 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:33209
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.62.keytab
--principal=oryx/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:44247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:33209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:14.693471 3497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:14.693679 3497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:14.693710 3497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:14.694832 3497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:00:14.694872 3497 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 23:00:14.694890 3497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:14.694905 3497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:14.694921 3497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:00:14.694944 3497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:00:14.696444 3497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:44247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:33209
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.62.keytab
--principal=oryx/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:33209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:14.696689 3497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:14.696885 3497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:14.701558 3505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:14.701584 3508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:14.701642 3497 server_base.cc:1034] running on GCE node
W20260808 23:00:14.701607 3504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:14.701927 3497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:14.702194 3497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:14.703361 3497 hybrid_clock.cc:648] HybridClock initialized: now 1786230014703337 us; error 32 us; skew 500 ppm
Aug 08 23:00:14 dist-test-slave-f0tn krb5kdc[3475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230014, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:14.705744 3497 init.cc:377] Logged in from keytab as oryx/127.23.194.62@KRBTEST.COM (short username oryx)
I20260808 23:00:14.706182 3497 webserver.cc:460] Webserver started at http://127.23.194.62:42849/ using document root <none> and password file <none>
I20260808 23:00:14.706400 3497 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:14.706449 3497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:14.706578 3497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:14.707619 3497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "eb9422677c124ee5b7d5fc3f97515bd7"
format_stamp: "Formatted at 2026-08-08 23:00:14 on dist-test-slave-f0tn"
server_key: "55147f5d532fd68fbf44be13c3e9bb24"
server_key_iv: "91a3ce8a342adc62e9f53f66c7c156cc"
server_key_version: "encryptionkey@0"
I20260808 23:00:14.707952 3497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "eb9422677c124ee5b7d5fc3f97515bd7"
format_stamp: "Formatted at 2026-08-08 23:00:14 on dist-test-slave-f0tn"
server_key: "55147f5d532fd68fbf44be13c3e9bb24"
server_key_iv: "91a3ce8a342adc62e9f53f66c7c156cc"
server_key_version: "encryptionkey@0"
I20260808 23:00:14.709187 3497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:00:14.709906 3517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:14.710103 3497 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:14.710179 3497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "eb9422677c124ee5b7d5fc3f97515bd7"
format_stamp: "Formatted at 2026-08-08 23:00:14 on dist-test-slave-f0tn"
server_key: "55147f5d532fd68fbf44be13c3e9bb24"
server_key_iv: "91a3ce8a342adc62e9f53f66c7c156cc"
server_key_version: "encryptionkey@0"
I20260808 23:00:14.710245 3497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:14.724033 3497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:14.725826 3497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:14.725966 3497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:14.730361 3497 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:33209
I20260808 23:00:14.730396 3632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:33209 every 8 connection(s)
I20260808 23:00:14.730701 3497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:00:14.731367 3634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:14.733712 3634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7: Bootstrap starting.
I20260808 23:00:14.734272 3634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:14.734537 3634 log.cc:824] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:14.735160 3634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7: No bootstrap required, opened a new log
I20260808 23:00:14.737169 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 3497
I20260808 23:00:14.737130 3634 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9422677c124ee5b7d5fc3f97515bd7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 33209 } }
I20260808 23:00:14.737254 3634 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:14.737270 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:00:14.737293 3634 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb9422677c124ee5b7d5fc3f97515bd7, State: Initialized, Role: FOLLOWER
I20260808 23:00:14.737484 24328 external_mini_cluster.cc:1428] Setting key 7f3e55777905fca5956e9439e9c3910e
I20260808 23:00:14.737458 3634 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9422677c124ee5b7d5fc3f97515bd7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 33209 } }
I20260808 23:00:14.737530 3634 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:00:14.737568 3634 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:00:14.737612 3634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:14.738253 3634 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9422677c124ee5b7d5fc3f97515bd7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 33209 } }
I20260808 23:00:14.738353 3634 leader_election.cc:302] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb9422677c124ee5b7d5fc3f97515bd7; no voters:
I20260808 23:00:14.738549 3634 leader_election.cc:288] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:00:14.738586 3639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:14.738770 3639 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7 [term 1 LEADER]: Becoming Leader. State: Replica: eb9422677c124ee5b7d5fc3f97515bd7, State: Running, Role: LEADER
I20260808 23:00:14.738813 3634 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:00:14.740092 3639 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9422677c124ee5b7d5fc3f97515bd7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 33209 } }
W20260808 23:00:14.741412 3659 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 23:00:14.740694 3640 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb9422677c124ee5b7d5fc3f97515bd7. Latest consensus state: current_term: 1 leader_uuid: "eb9422677c124ee5b7d5fc3f97515bd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9422677c124ee5b7d5fc3f97515bd7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 33209 } } }
W20260808 23:00:14.741462 3659 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:00:14.740821 3642 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb9422677c124ee5b7d5fc3f97515bd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9422677c124ee5b7d5fc3f97515bd7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 33209 } } }
I20260808 23:00:14.741487 3640 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:14.741499 3642 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:14.741688 3666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:00:14.741858 3666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:00:14.743362 3666 catalog_manager.cc:1347] Generated new cluster ID: 3e4e6164a4784065bddd12f15cf267c5
I20260808 23:00:14.743415 3666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 23:00:14 dist-test-slave-f0tn krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM
I20260808 23:00:14.748330 3638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:14.737946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38678 (local address 127.23.194.62:33209)
0808 23:00:14.738108 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:14.738116 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:14.738160 (+ 44us) server_negotiation.cc:407] Connection header received
0808 23:00:14.738249 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:14.738254 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:14.738293 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:14.738414 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:14.739146 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:14.740539 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:14.741406 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:14.742011 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:14.742110 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:14.744899 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:14.744903 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:14.744903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:14.744915 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:14.746796 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:14.747199 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:14.747202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:14.747202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:14.747307 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:14.747502 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:14.747503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:14.747503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:14.747764 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:14.747903 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:14.748013 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 23:00:14.748144 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":75,"threads_started":1}
I20260808 23:00:14.752393 3666 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:00:14.753062 3666 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:00:14.759214 3666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7: Generated new TSK 0
I20260808 23:00:14.759367 3666 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.1.keytab.
Entry for principal oryx/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.1.keytab.
I20260808 23:00:14.784961 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:33209
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.1.keytab
--principal=oryx/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:44247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:14.865361 3673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:14.865599 3673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:14.865623 3673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:14.866758 3673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:14.866798 3673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:14.866817 3673 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:00:14.866833 3673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:00:14.868366 3673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:44247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.1.keytab
--principal=oryx/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.194.62:33209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:14.868610 3673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:14.868844 3673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:14.873624 3681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:14.873629 3680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:14.873634 3684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:14.873704 3673 server_base.cc:1034] running on GCE node
I20260808 23:00:14.874064 3673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:14.874305 3673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:14.875454 3673 hybrid_clock.cc:648] HybridClock initialized: now 1786230014875441 us; error 32 us; skew 500 ppm
Aug 08 23:00:14 dist-test-slave-f0tn krb5kdc[3475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230014, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:14.877915 3673 init.cc:377] Logged in from keytab as oryx/127.23.194.1@KRBTEST.COM (short username oryx)
I20260808 23:00:14.878346 3673 webserver.cc:460] Webserver started at http://127.23.194.1:37085/ using document root <none> and password file <none>
I20260808 23:00:14.878552 3673 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:14.878600 3673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:14.878726 3673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:14.879714 3673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "9c2ec16fd44742b788f43a95215c08e7"
format_stamp: "Formatted at 2026-08-08 23:00:14 on dist-test-slave-f0tn"
server_key: "4452930220b833d3e28a09faabc468a4"
server_key_iv: "a91090002b9207f77e4c5ece25c192a6"
server_key_version: "encryptionkey@0"
I20260808 23:00:14.880036 3673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "9c2ec16fd44742b788f43a95215c08e7"
format_stamp: "Formatted at 2026-08-08 23:00:14 on dist-test-slave-f0tn"
server_key: "4452930220b833d3e28a09faabc468a4"
server_key_iv: "a91090002b9207f77e4c5ece25c192a6"
server_key_version: "encryptionkey@0"
I20260808 23:00:14.881201 3673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:00:14.881891 3692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:14.882056 3673 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:14.882139 3673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "9c2ec16fd44742b788f43a95215c08e7"
format_stamp: "Formatted at 2026-08-08 23:00:14 on dist-test-slave-f0tn"
server_key: "4452930220b833d3e28a09faabc468a4"
server_key_iv: "a91090002b9207f77e4c5ece25c192a6"
server_key_version: "encryptionkey@0"
I20260808 23:00:14.882203 3673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:14.904129 3673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:14.905830 3673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:14.905961 3673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:14.906548 3673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:14.906598 3673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:14.906627 3673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:14.906669 3673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 23:00:14 dist-test-slave-f0tn krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230014, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM
I20260808 23:00:14.913537 3673 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:43967
I20260808 23:00:14.913601 3873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:43967 every 8 connection(s)
I20260808 23:00:14.913928 3673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:00:14.916146 3638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:14.906899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:56133 (local address 127.23.194.62:33209)
0808 23:00:14.906958 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:14.906960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:14.907191 (+ 231us) server_negotiation.cc:407] Connection header received
0808 23:00:14.907834 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:14.907835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:14.907849 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:14.907891 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:14.908710 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:14.909490 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:14.910960 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:14.911271 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:14.911346 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:14.913835 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:14.913836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:14.913837 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:14.913844 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:14.915144 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:14.915477 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:14.915478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:14.915478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:14.915537 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:14.915694 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:14.915694 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:14.915694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:14.915785 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:14.915829 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:14.916021 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 23:00:14.916082 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 23:00:14.916312 3707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:14.906887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:33209 (local address 127.23.194.1:56133)
0808 23:00:14.907050 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:14.907097 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:14.907732 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:14.907965 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:14.907970 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:14.908322 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:14.908615 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:14.908622 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:14.909580 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:14.909582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:14.910865 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:14.910868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:14.911371 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:14.911372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:14.911528 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:14.912002 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:14.912013 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:14.913726 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:14.915209 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:14.915215 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:14.915216 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:14.915419 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:14.915572 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:14.915573 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:14.915573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:14.915656 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:14.915825 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:14.915828 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:14.915929 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 23:00:14.916002 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 23:00:14.916113 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
I20260808 23:00:14.916747 3874 heartbeater.cc:347] Connected to a master server at 127.23.194.62:33209
I20260808 23:00:14.916826 3874 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:14.916987 3874 heartbeater.cc:511] Master 127.23.194.62:33209 requested a full tablet report, sending...
I20260808 23:00:14.917498 3557 ts_manager.cc:194] Registered new tserver with Master: 9c2ec16fd44742b788f43a95215c08e7 (127.23.194.1:43967)
I20260808 23:00:14.918160 3557 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:56133
I20260808 23:00:14.919925 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 3673
I20260808 23:00:14.920053 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:00:14.920274 24328 external_mini_cluster.cc:1428] Setting key 6e78b9280a9219f9c8a023d081ee428e
I20260808 23:00:14.923444 3638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:14.917734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:34505 (local address 127.23.194.62:33209)
0808 23:00:14.917769 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:14.917770 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:14.918434 (+ 664us) server_negotiation.cc:407] Connection header received
0808 23:00:14.918495 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:14.918496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:14.918505 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:14.918557 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:14.918828 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:14.919351 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:14.920219 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:14.920421 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:14.920471 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:14.921241 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:14.921242 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:14.921243 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:14.921249 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:14.922444 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:14.922773 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:14.922774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:14.922775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:14.922829 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:14.922991 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:14.922992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:14.922992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:14.923093 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:14.923146 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:14.923280 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 23:00:14.923351 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 23:00:14.923457 3880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:14.917680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:33209 (local address 127.23.194.1:34505)
0808 23:00:14.918375 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:14.918386 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:14.918444 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:14.918559 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:14.918559 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:14.918716 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:14.918767 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:14.918768 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:14.919455 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:14.919456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:14.920132 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:14.920135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:14.920477 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:14.920478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:14.920551 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:14.920915 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:14.920919 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:14.921172 (+ 253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:14.922529 (+ 1357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:14.922530 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:14.922530 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:14.922708 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:14.922885 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:14.922885 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:14.922886 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:14.922941 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:14.923147 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:14.923148 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:14.923192 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 23:00:14.923253 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 23:00:14.923351 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for oryx/127.23.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.2.keytab.
Entry for principal oryx/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.2.keytab.
I20260808 23:00:14.944186 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:33209
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.2.keytab
--principal=oryx/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:44247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:15.025048 3884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:15.025252 3884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:15.025275 3884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:15.026400 3884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:15.026434 3884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:15.026453 3884 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:00:15.026468 3884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:00:15.027925 3884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:44247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.2.keytab
--principal=oryx/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.194.62:33209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:15.028110 3884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:15.028288 3884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:15.033080 3892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:15.033079 3897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:15.033298 3891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:15.033279 3884 server_base.cc:1034] running on GCE node
I20260808 23:00:15.033536 3884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:15.033746 3884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:15.034878 3884 hybrid_clock.cc:648] HybridClock initialized: now 1786230015034865 us; error 28 us; skew 500 ppm
Aug 08 23:00:15 dist-test-slave-f0tn krb5kdc[3475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230015, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:15.037307 3884 init.cc:377] Logged in from keytab as oryx/127.23.194.2@KRBTEST.COM (short username oryx)
I20260808 23:00:15.037738 3884 webserver.cc:460] Webserver started at http://127.23.194.2:45269/ using document root <none> and password file <none>
I20260808 23:00:15.037955 3884 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:15.038003 3884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:15.038105 3884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:15.039127 3884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "a0f3243b474d4e25866098bc36f944c6"
format_stamp: "Formatted at 2026-08-08 23:00:15 on dist-test-slave-f0tn"
server_key: "b175e80bbb8536280d79575130400315"
server_key_iv: "d8110baf958a57f11130f775fbe3ff2e"
server_key_version: "encryptionkey@0"
I20260808 23:00:15.039460 3884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "a0f3243b474d4e25866098bc36f944c6"
format_stamp: "Formatted at 2026-08-08 23:00:15 on dist-test-slave-f0tn"
server_key: "b175e80bbb8536280d79575130400315"
server_key_iv: "d8110baf958a57f11130f775fbe3ff2e"
server_key_version: "encryptionkey@0"
I20260808 23:00:15.040714 3884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:00:15.041428 3907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:15.041627 3884 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:00:15.041718 3884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "a0f3243b474d4e25866098bc36f944c6"
format_stamp: "Formatted at 2026-08-08 23:00:15 on dist-test-slave-f0tn"
server_key: "b175e80bbb8536280d79575130400315"
server_key_iv: "d8110baf958a57f11130f775fbe3ff2e"
server_key_version: "encryptionkey@0"
I20260808 23:00:15.041805 3884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:15.060576 3884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:15.062590 3884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:15.062716 3884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:15.063277 3884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:15.063319 3884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:15.063341 3884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:15.063354 3884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 23:00:15 dist-test-slave-f0tn krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230015, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM
I20260808 23:00:15.069849 3884 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:44439
I20260808 23:00:15.069921 4091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:44439 every 8 connection(s)
I20260808 23:00:15.070272 3884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:00:15.072402 3638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.063728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:36757 (local address 127.23.194.62:33209)
0808 23:00:15.063795 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.063798 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.063963 (+ 165us) server_negotiation.cc:407] Connection header received
0808 23:00:15.064434 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.064435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.064447 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.064529 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:15.065325 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.066101 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.067095 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.067378 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.067454 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.069809 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:15.069810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:15.069811 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:15.069818 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:15.071196 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:15.071539 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:15.071540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:15.071540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:15.071593 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:15.071804 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:15.071804 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:15.071805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:15.071915 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:15.071970 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.072224 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.072312 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 23:00:15.072541 3920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.063687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:33209 (local address 127.23.194.2:36757)
0808 23:00:15.063837 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:15.063877 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:15.064346 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:15.064611 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:15.064615 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:15.064944 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:15.065213 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.065219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.066186 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.066187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.067027 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.067030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.067470 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.067471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.067611 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.068145 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:15.068156 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:15.069726 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:15.071266 (+ 1540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:15.071271 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:15.071273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:15.071475 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:15.071645 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:15.071646 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:15.071646 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:15.071750 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:15.071967 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:15.071973 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:15.072101 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 23:00:15.072195 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 23:00:15.072341 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":83,"threads_started":1}
I20260808 23:00:15.072990 4093 heartbeater.cc:347] Connected to a master server at 127.23.194.62:33209
I20260808 23:00:15.073081 4093 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:15.073246 4093 heartbeater.cc:511] Master 127.23.194.62:33209 requested a full tablet report, sending...
I20260808 23:00:15.073673 3557 ts_manager.cc:194] Registered new tserver with Master: a0f3243b474d4e25866098bc36f944c6 (127.23.194.2:44439)
I20260808 23:00:15.074123 3557 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:36757
I20260808 23:00:15.078778 3638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.073956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:45175 (local address 127.23.194.62:33209)
0808 23:00:15.073992 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.073993 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.074325 (+ 332us) server_negotiation.cc:407] Connection header received
0808 23:00:15.074369 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.074370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.074380 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.074432 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:15.074782 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.075230 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.075831 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.075993 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.076032 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.076764 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:15.076765 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:15.076765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:15.076769 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:15.077961 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:15.078255 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:15.078256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:15.078256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:15.078298 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:15.078416 (+ 118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:15.078417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:15.078417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:15.078532 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:15.078580 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.078668 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.078722 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 23:00:15.078804 4103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.073910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:33209 (local address 127.23.194.2:45175)
0808 23:00:15.074273 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:15.074283 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:15.074334 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:15.074461 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:15.074462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:15.074648 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:15.074718 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.074720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.075302 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.075303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.075776 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.075777 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.076041 (+ 264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.076041 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.076084 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.076446 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:15.076451 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:15.076708 (+ 257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:15.078042 (+ 1334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:15.078043 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:15.078043 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:15.078196 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:15.078339 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:15.078339 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:15.078340 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:15.078384 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:15.078578 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:15.078578 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:15.078609 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 23:00:15.078652 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 23:00:15.078734 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":68,"threads_started":1}
I20260808 23:00:15.079239 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 3884
I20260808 23:00:15.079341 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:00:15.079527 24328 external_mini_cluster.cc:1428] Setting key 9b5fc22191af1c0227537d7b1a6a293f
WARNING: no policy specified for oryx/127.23.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.3.keytab.
Entry for principal oryx/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.3.keytab.
I20260808 23:00:15.103832 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:33209
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.3.keytab
--principal=oryx/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:44247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:15.183889 4107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:15.184101 4107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:15.184134 4107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:15.185281 4107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:15.185319 4107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:15.185346 4107 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:00:15.185370 4107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:00:15.186882 4107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:44247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.3.keytab
--principal=oryx/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.194.62:33209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:15.187114 4107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:15.187321 4107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:15.191957 4114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:15.191979 4112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:15.191962 4119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:15.192167 4107 server_base.cc:1034] running on GCE node
I20260808 23:00:15.192363 4107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:15.192574 4107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:15.193702 4107 hybrid_clock.cc:648] HybridClock initialized: now 1786230015193688 us; error 29 us; skew 500 ppm
Aug 08 23:00:15 dist-test-slave-f0tn krb5kdc[3475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230015, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:15.199687 4107 init.cc:377] Logged in from keytab as oryx/127.23.194.3@KRBTEST.COM (short username oryx)
I20260808 23:00:15.200088 4107 webserver.cc:460] Webserver started at http://127.23.194.3:36527/ using document root <none> and password file <none>
I20260808 23:00:15.200304 4107 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:15.200350 4107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:15.200472 4107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:15.201504 4107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "5de46e59381f438b9436e35be05cb093"
format_stamp: "Formatted at 2026-08-08 23:00:15 on dist-test-slave-f0tn"
server_key: "cbb3dc67ef72c293737c0b28d45a11d4"
server_key_iv: "e0d0cd49445798be1bd6f969899e4cc7"
server_key_version: "encryptionkey@0"
I20260808 23:00:15.201887 4107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "5de46e59381f438b9436e35be05cb093"
format_stamp: "Formatted at 2026-08-08 23:00:15 on dist-test-slave-f0tn"
server_key: "cbb3dc67ef72c293737c0b28d45a11d4"
server_key_iv: "e0d0cd49445798be1bd6f969899e4cc7"
server_key_version: "encryptionkey@0"
I20260808 23:00:15.203728 4107 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 23:00:15.204344 4126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:15.204486 4107 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:15.204568 4107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "5de46e59381f438b9436e35be05cb093"
format_stamp: "Formatted at 2026-08-08 23:00:15 on dist-test-slave-f0tn"
server_key: "cbb3dc67ef72c293737c0b28d45a11d4"
server_key_iv: "e0d0cd49445798be1bd6f969899e4cc7"
server_key_version: "encryptionkey@0"
I20260808 23:00:15.204629 4107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:15.221618 4107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:15.223685 4107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:15.223822 4107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:15.224355 4107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:15.224395 4107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:15.224426 4107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:15.224444 4107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 23:00:15 dist-test-slave-f0tn krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230015, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM
I20260808 23:00:15.231325 4107 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:37529
I20260808 23:00:15.231637 4308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:37529 every 8 connection(s)
I20260808 23:00:15.231714 4107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 23:00:15.233357 3638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.224705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:43297 (local address 127.23.194.62:33209)
0808 23:00:15.224768 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.224771 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.224972 (+ 201us) server_negotiation.cc:407] Connection header received
0808 23:00:15.225493 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.225495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.225510 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.225573 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:15.226183 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.226678 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.227679 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.227870 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.227926 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.230643 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:15.230645 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:15.230645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:15.230660 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:15.232110 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:15.232461 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:15.232463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:15.232463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:15.232523 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:15.232736 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:15.232737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:15.232738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:15.232871 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:15.232931 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.233185 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.233268 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 23:00:15.233541 4141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.224690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:33209 (local address 127.23.194.3:43297)
0808 23:00:15.224846 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:15.224887 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:15.225355 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:15.225638 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:15.225642 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:15.225909 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:15.226093 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.226098 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.226759 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.226760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.227585 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.227588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.227939 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.227940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.228113 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.228567 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:15.228576 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:15.230549 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:15.232177 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:15.232181 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:15.232184 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:15.232395 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:15.232575 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:15.232575 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:15.232576 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:15.232680 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:15.232922 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:15.232929 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:15.233062 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 23:00:15.233150 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 23:00:15.233311 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260808 23:00:15.233997 4309 heartbeater.cc:347] Connected to a master server at 127.23.194.62:33209
I20260808 23:00:15.234077 4309 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:15.234236 4309 heartbeater.cc:511] Master 127.23.194.62:33209 requested a full tablet report, sending...
I20260808 23:00:15.234601 3559 ts_manager.cc:194] Registered new tserver with Master: 5de46e59381f438b9436e35be05cb093 (127.23.194.3:37529)
I20260808 23:00:15.235038 3559 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:43297
I20260808 23:00:15.238692 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 4107
I20260808 23:00:15.238827 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:00:15.239017 24328 external_mini_cluster.cc:1428] Setting key e199f64dc558e8b959562102fe703bfe
I20260808 23:00:15.239830 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:00:15.239913 3638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.235002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:53383 (local address 127.23.194.62:33209)
0808 23:00:15.235039 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.235041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.235244 (+ 203us) server_negotiation.cc:407] Connection header received
0808 23:00:15.235286 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.235286 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.235296 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.235346 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:15.235669 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.236107 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.236705 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.236867 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.236907 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.237620 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:15.237621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:15.237621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:15.237627 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:15.238836 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:15.239164 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:15.239165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:15.239165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:15.239222 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:15.239413 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:15.239413 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:15.239414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:15.239546 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:15.239607 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.239764 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.239855 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 23:00:15.239967 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.234943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:33209 (local address 127.23.194.3:53383)
0808 23:00:15.235193 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:15.235202 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:15.235252 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:15.235351 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:15.235352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:15.235543 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:15.235604 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.235606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.236185 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.236186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.236648 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.236650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.236910 (+ 260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.236911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.236951 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.237257 (+ 306us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:15.237261 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:15.237548 (+ 287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:15.238921 (+ 1373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:15.238922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:15.238922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:15.239092 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:15.239282 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:15.239283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:15.239283 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:15.239356 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:15.239611 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:15.239612 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:15.239666 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 23:00:15.239736 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 23:00:15.239877 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":89,"threads_started":1}
Aug 08 23:00:15 dist-test-slave-f0tn krb5kdc[3475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230015, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 23:00:15 dist-test-slave-f0tn krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:15 dist-test-slave-f0tn krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 23:00:15.260669 3638 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:15.256013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38684 (local address 127.23.194.62:33209)
0808 23:00:15.256062 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.256064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.256090 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:00:15.256112 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.256113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.256122 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.256162 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:15.256649 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.257114 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.257798 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.257976 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.258025 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.260589 (+ 2564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38684: BlockingRecv error: recv got EOF from 127.0.0.1:38684 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 08 23:00:15 dist-test-slave-f0tn krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230015, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM
I20260808 23:00:15.267163 3638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.261449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38696 (local address 127.23.194.62:33209)
0808 23:00:15.261498 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.261500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.261583 (+ 83us) server_negotiation.cc:407] Connection header received
0808 23:00:15.261645 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.261646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.261666 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.261706 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:15.262150 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.262592 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.263231 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.263403 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.263454 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.265303 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:15.265304 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:15.265304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:15.265311 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:15.266380 (+ 1069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:15.266663 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:15.266663 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:15.266664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:15.266709 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:15.266828 (+ 119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:15.266828 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:15.266828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:15.266913 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:15.266975 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.267043 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.267100 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 23:00:15.268565 3559 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:00:15.270188 3559 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:00:15.276289 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.272256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:43967 (local address 127.0.0.1:55212)
0808 23:00:15.272425 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:15.272463 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:15.272522 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:15.272834 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:15.272836 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:15.272839 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:15.272898 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.272900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.274081 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.274082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.275238 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.275240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.275990 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.275991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.276078 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.276094 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 23:00:15.276160 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 23:00:15.276215 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":58,"threads_started":1}
I20260808 23:00:15.276319 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.272366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55212 (local address 127.23.194.1:43967)
0808 23:00:15.272697 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.272701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.272709 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:00:15.272739 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.272740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.272752 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.272820 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:15.273016 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.274009 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.275332 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.275920 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.275980 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.276063 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.276188 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.276228 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":78,"threads_started":1}
I20260808 23:00:15.276619 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.272495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43354 (local address 127.23.194.3:37529)
0808 23:00:15.272689 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.272694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.272704 (+ 10us) server_negotiation.cc:407] Connection header received
0808 23:00:15.272743 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.272745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.272761 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.272915 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:15.273014 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.274363 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.275387 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.276006 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.276073 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.276140 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.276485 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.276526 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":78,"threads_started":1}
I20260808 23:00:15.276413 4342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.272265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:37529 (local address 127.0.0.1:43354)
0808 23:00:15.272530 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:15.272541 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:15.272596 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:15.272913 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:15.272914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:15.272916 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:15.272955 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.272955 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.274456 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.274457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.275316 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.275318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.276188 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.276189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.276250 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.276255 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:00:15.276302 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 23:00:15.276341 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":36,"thread_start_us":91,"threads_started":1}
I20260808 23:00:15.276684 4346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.272268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:44439 (local address 127.0.0.1:46010)
0808 23:00:15.272766 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:15.272774 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:15.272821 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:15.273104 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:15.273105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:15.273108 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:15.273148 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.273149 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.274460 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.274461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.275533 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.275536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.276464 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.276465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.276520 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.276524 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:00:15.276580 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 23:00:15.276616 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":10,"thread_start_us":71,"threads_started":1}
I20260808 23:00:15.276988 4353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.272511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46010 (local address 127.23.194.2:44439)
0808 23:00:15.272722 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.272726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.272895 (+ 169us) server_negotiation.cc:407] Connection header received
0808 23:00:15.272934 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.272935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.272949 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.273014 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:15.273248 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.274362 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.275612 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.276308 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.276362 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.276429 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.276612 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.276893 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":70,"threads_started":1}
I20260808 23:00:15.277268 3765 tablet_service.cc:1467] Processing CreateTablet for tablet f4213435c4dd400a8480f87c3efcb1d4 (DEFAULT_TABLE table=test-table [id=907ba427f0884c07b8432f7a98ef2498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:15.277655 3765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4213435c4dd400a8480f87c3efcb1d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:15.277539 4202 tablet_service.cc:1467] Processing CreateTablet for tablet f4213435c4dd400a8480f87c3efcb1d4 (DEFAULT_TABLE table=test-table [id=907ba427f0884c07b8432f7a98ef2498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:15.277633 3995 tablet_service.cc:1467] Processing CreateTablet for tablet f4213435c4dd400a8480f87c3efcb1d4 (DEFAULT_TABLE table=test-table [id=907ba427f0884c07b8432f7a98ef2498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:15.277884 4202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4213435c4dd400a8480f87c3efcb1d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:15.277916 3995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4213435c4dd400a8480f87c3efcb1d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:15.280704 4354 tablet_bootstrap.cc:492] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7: Bootstrap starting.
I20260808 23:00:15.281232 4354 tablet_bootstrap.cc:654] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:15.281304 4356 tablet_bootstrap.cc:492] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093: Bootstrap starting.
I20260808 23:00:15.281594 4354 log.cc:824] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:15.281906 4357 tablet_bootstrap.cc:492] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6: Bootstrap starting.
I20260808 23:00:15.281966 4356 tablet_bootstrap.cc:654] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:15.282338 4354 tablet_bootstrap.cc:492] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7: No bootstrap required, opened a new log
I20260808 23:00:15.282322 4356 log.cc:824] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:15.282407 4354 ts_tablet_manager.cc:1397] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 23:00:15.282420 4357 tablet_bootstrap.cc:654] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:15.282721 4357 log.cc:824] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:15.283154 4356 tablet_bootstrap.cc:492] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093: No bootstrap required, opened a new log
I20260808 23:00:15.283218 4356 ts_tablet_manager.cc:1397] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:00:15.283335 4357 tablet_bootstrap.cc:492] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6: No bootstrap required, opened a new log
I20260808 23:00:15.283377 4357 ts_tablet_manager.cc:1397] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:00:15.284248 4354 raft_consensus.cc:348] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } }
I20260808 23:00:15.284403 4354 raft_consensus.cc:374] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:15.284438 4354 raft_consensus.cc:729] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2ec16fd44742b788f43a95215c08e7, State: Initialized, Role: FOLLOWER
I20260808 23:00:15.284545 4354 consensus_queue.cc:260] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } }
I20260808 23:00:15.284816 4354 ts_tablet_manager.cc:1428] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 23:00:15.284844 3874 heartbeater.cc:503] Master 127.23.194.62:33209 was elected leader, sending a full tablet report...
I20260808 23:00:15.285096 4356 raft_consensus.cc:348] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } }
I20260808 23:00:15.285248 4356 raft_consensus.cc:374] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:15.285291 4356 raft_consensus.cc:729] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5de46e59381f438b9436e35be05cb093, State: Initialized, Role: FOLLOWER
I20260808 23:00:15.285419 4356 consensus_queue.cc:260] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } }
I20260808 23:00:15.285686 4356 ts_tablet_manager.cc:1428] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 23:00:15.285758 4309 heartbeater.cc:503] Master 127.23.194.62:33209 was elected leader, sending a full tablet report...
I20260808 23:00:15.285856 4357 raft_consensus.cc:348] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } }
I20260808 23:00:15.286002 4357 raft_consensus.cc:374] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:15.286049 4357 raft_consensus.cc:729] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f3243b474d4e25866098bc36f944c6, State: Initialized, Role: FOLLOWER
I20260808 23:00:15.286221 4357 consensus_queue.cc:260] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } }
I20260808 23:00:15.286481 4357 ts_tablet_manager.cc:1428] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 23:00:15.286556 4093 heartbeater.cc:503] Master 127.23.194.62:33209 was elected leader, sending a full tablet report...
W20260808 23:00:15.320819 4097 tablet.cc:2369] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:00:15.414986 3875 tablet.cc:2369] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:00:15.451820 4368 raft_consensus.cc:482] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:00:15.451947 4368 raft_consensus.cc:504] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } }
I20260808 23:00:15.452286 4368 leader_election.cc:288] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2ec16fd44742b788f43a95215c08e7 (127.23.194.1:43967), 5de46e59381f438b9436e35be05cb093 (127.23.194.3:37529)
I20260808 23:00:15.455108 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.452572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:46107 (local address 127.23.194.1:43967)
0808 23:00:15.452622 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.452624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.452786 (+ 162us) server_negotiation.cc:407] Connection header received
0808 23:00:15.452831 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.452832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.452841 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.452891 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:15.453028 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.453482 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.454430 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.454786 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.454845 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.454855 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.454997 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.455036 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 23:00:15.455112 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.452601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:51675 (local address 127.23.194.3:37529)
0808 23:00:15.452644 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.452646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.452655 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:00:15.452685 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.452686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.452695 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.452761 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:15.452912 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.453466 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.454438 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.454790 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.454855 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.454863 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.454997 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.455048 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 23:00:15.455117 3920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.452502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:37529 (local address 127.23.194.2:51675)
0808 23:00:15.452563 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:15.452575 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:15.452626 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:15.452771 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:15.452772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:15.452774 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:15.452848 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.452850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.453543 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.453544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.454350 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.454353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.454867 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.454868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.454935 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.454937 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 23:00:15.454983 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 23:00:15.455042 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":87}
I20260808 23:00:15.455130 4370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.452529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:43967 (local address 127.23.194.2:46107)
0808 23:00:15.452712 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:15.452722 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:15.452777 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:15.452900 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:15.452900 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:15.452912 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:15.452971 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.452972 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.453551 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.453552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.454344 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.454346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.454861 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.454861 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.454933 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.454936 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:00:15.454983 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 23:00:15.455034 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":2,"thread_start_us":115,"threads_started":1}
I20260808 23:00:15.455567 4231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4213435c4dd400a8480f87c3efcb1d4" candidate_uuid: "a0f3243b474d4e25866098bc36f944c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de46e59381f438b9436e35be05cb093" is_pre_election: true
I20260808 23:00:15.455619 3792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4213435c4dd400a8480f87c3efcb1d4" candidate_uuid: "a0f3243b474d4e25866098bc36f944c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2ec16fd44742b788f43a95215c08e7" is_pre_election: true
I20260808 23:00:15.455709 4231 raft_consensus.cc:2454] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0f3243b474d4e25866098bc36f944c6 in term 0.
I20260808 23:00:15.455763 3792 raft_consensus.cc:2454] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0f3243b474d4e25866098bc36f944c6 in term 0.
I20260808 23:00:15.455953 3911 leader_election.cc:302] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2ec16fd44742b788f43a95215c08e7, a0f3243b474d4e25866098bc36f944c6; no voters:
I20260808 23:00:15.456073 4368 raft_consensus.cc:2781] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:00:15.456146 4368 raft_consensus.cc:482] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:00:15.456166 4368 raft_consensus.cc:3037] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:15.457074 4368 raft_consensus.cc:504] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } }
I20260808 23:00:15.457198 4368 leader_election.cc:288] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [CANDIDATE]: Term 1 election: Requested vote from peers 9c2ec16fd44742b788f43a95215c08e7 (127.23.194.1:43967), 5de46e59381f438b9436e35be05cb093 (127.23.194.3:37529)
I20260808 23:00:15.457348 3792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4213435c4dd400a8480f87c3efcb1d4" candidate_uuid: "a0f3243b474d4e25866098bc36f944c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2ec16fd44742b788f43a95215c08e7"
I20260808 23:00:15.457366 4231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4213435c4dd400a8480f87c3efcb1d4" candidate_uuid: "a0f3243b474d4e25866098bc36f944c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de46e59381f438b9436e35be05cb093"
I20260808 23:00:15.457451 3792 raft_consensus.cc:3037] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:15.457468 4231 raft_consensus.cc:3037] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:15.458237 3792 raft_consensus.cc:2454] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0f3243b474d4e25866098bc36f944c6 in term 1.
I20260808 23:00:15.458276 4231 raft_consensus.cc:2454] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0f3243b474d4e25866098bc36f944c6 in term 1.
I20260808 23:00:15.458388 3911 leader_election.cc:302] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2ec16fd44742b788f43a95215c08e7, a0f3243b474d4e25866098bc36f944c6; no voters:
I20260808 23:00:15.458475 4368 raft_consensus.cc:2781] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:15.458631 4368 raft_consensus.cc:686] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [term 1 LEADER]: Becoming Leader. State: Replica: a0f3243b474d4e25866098bc36f944c6, State: Running, Role: LEADER
I20260808 23:00:15.458727 4368 consensus_queue.cc:237] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } }
I20260808 23:00:15.459463 3559 catalog_manager.cc:5495] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0f3243b474d4e25866098bc36f944c6 (127.23.194.2). New cstate: current_term: 1 leader_uuid: "a0f3243b474d4e25866098bc36f944c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } health_report { overall_health: HEALTHY } } }
W20260808 23:00:15.482271 4310 tablet.cc:2369] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:00:15.504630 4353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.502444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46014 (local address 127.23.194.2:44439)
0808 23:00:15.502501 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.502503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.502511 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:00:15.502557 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.502558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.502575 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.502629 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:00:15.502768 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.503232 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.503953 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.504131 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.504189 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.504340 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:00:15.504437 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:00:15.504502 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.504540 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.504559 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 23:00:15.506328 4231 raft_consensus.cc:1261] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093 [term 1 FOLLOWER]: Refusing update from remote peer a0f3243b474d4e25866098bc36f944c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:00:15.506328 3792 raft_consensus.cc:1261] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7 [term 1 FOLLOWER]: Refusing update from remote peer a0f3243b474d4e25866098bc36f944c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:00:15.506651 4371 consensus_queue.cc:1035] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:15.506778 4368 consensus_queue.cc:1035] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:15.508955 4377 mvcc.cc:205] Tried to move back new op lower bound from 7316398143511240704 to 7316398143319416832. Current Snapshot: MvccSnapshot[applied={T|T < 7316398143511240704}]
I20260808 23:00:15.509707 4378 mvcc.cc:205] Tried to move back new op lower bound from 7316398143511240704 to 7316398143319416832. Current Snapshot: MvccSnapshot[applied={T|T < 7316398143511240704}]
I20260808 23:00:15.511243 3557 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:00:15.511332 3557 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:00:15.512001 3557 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7: Sending DeleteTablet for 3 replicas of tablet f4213435c4dd400a8480f87c3efcb1d4
I20260808 23:00:15.512356 3765 tablet_service.cc:1514] Processing DeleteTablet for tablet f4213435c4dd400a8480f87c3efcb1d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:15 UTC) from {username='oryx', principal='oryx/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:55212
I20260808 23:00:15.512384 4202 tablet_service.cc:1514] Processing DeleteTablet for tablet f4213435c4dd400a8480f87c3efcb1d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:15 UTC) from {username='oryx', principal='oryx/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:43354
I20260808 23:00:15.512604 4396 tablet_replica.cc:330] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093: stopping tablet replica
I20260808 23:00:15.512594 3995 tablet_service.cc:1514] Processing DeleteTablet for tablet f4213435c4dd400a8480f87c3efcb1d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:15 UTC) from {username='oryx', principal='oryx/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:46010
I20260808 23:00:15.512663 4394 tablet_replica.cc:330] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7: stopping tablet replica
I20260808 23:00:15.512768 4394 raft_consensus.cc:2229] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:00:15.512765 4396 raft_consensus.cc:2229] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:00:15.512773 3557 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 23:00:15.512912 4394 raft_consensus.cc:2258] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 23:00:15.512918 3557 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:00:15.513100 4396 raft_consensus.cc:2258] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:15.513643 4396 ts_tablet_manager.cc:1905] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:15.514056 4394 ts_tablet_manager.cc:1905] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:15.514225 3765 tablet_service.cc:1467] Processing CreateTablet for tablet 850114eb720f4c41b676635c961b6276 (DEFAULT_TABLE table=test-table [id=5cce6cd22c9a4c88a0a76138bf279dcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:15.514338 3765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 850114eb720f4c41b676635c961b6276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:15.515208 3995 tablet_service.cc:1467] Processing CreateTablet for tablet 850114eb720f4c41b676635c961b6276 (DEFAULT_TABLE table=test-table [id=5cce6cd22c9a4c88a0a76138bf279dcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:15.515322 3995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 850114eb720f4c41b676635c961b6276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:15.515504 4394 ts_tablet_manager.cc:1918] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:00:15.515540 4354 tablet_bootstrap.cc:492] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7: Bootstrap starting.
I20260808 23:00:15.515565 4394 log.cc:1196] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/wal/wals/f4213435c4dd400a8480f87c3efcb1d4
I20260808 23:00:15.515817 4394 ts_tablet_manager.cc:1939] T f4213435c4dd400a8480f87c3efcb1d4 P 9c2ec16fd44742b788f43a95215c08e7: Deleting consensus metadata
I20260808 23:00:15.515897 4354 tablet_bootstrap.cc:654] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:15.516330 4202 tablet_service.cc:1467] Processing CreateTablet for tablet 850114eb720f4c41b676635c961b6276 (DEFAULT_TABLE table=test-table [id=5cce6cd22c9a4c88a0a76138bf279dcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:15.516440 4354 tablet_bootstrap.cc:492] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7: No bootstrap required, opened a new log
I20260808 23:00:15.516476 4354 ts_tablet_manager.cc:1397] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 23:00:15.516628 4357 tablet_bootstrap.cc:492] T 850114eb720f4c41b676635c961b6276 P a0f3243b474d4e25866098bc36f944c6: Bootstrap starting.
I20260808 23:00:15.516616 3527 catalog_manager.cc:4841] TS 9c2ec16fd44742b788f43a95215c08e7 (127.23.194.1:43967): tablet f4213435c4dd400a8480f87c3efcb1d4 (table test-table [id=907ba427f0884c07b8432f7a98ef2498]) successfully deleted
I20260808 23:00:15.516779 4354 raft_consensus.cc:348] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.516845 4354 raft_consensus.cc:374] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:15.516865 4354 raft_consensus.cc:729] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2ec16fd44742b788f43a95215c08e7, State: Initialized, Role: FOLLOWER
I20260808 23:00:15.516906 4202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 850114eb720f4c41b676635c961b6276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:15.516902 4354 consensus_queue.cc:260] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.516986 4354 ts_tablet_manager.cc:1428] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 23:00:15.517017 4357 tablet_bootstrap.cc:654] T 850114eb720f4c41b676635c961b6276 P a0f3243b474d4e25866098bc36f944c6: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:15.517976 4356 tablet_bootstrap.cc:492] T 850114eb720f4c41b676635c961b6276 P 5de46e59381f438b9436e35be05cb093: Bootstrap starting.
I20260808 23:00:15.518108 4357 tablet_bootstrap.cc:492] T 850114eb720f4c41b676635c961b6276 P a0f3243b474d4e25866098bc36f944c6: No bootstrap required, opened a new log
I20260808 23:00:15.518122 4396 ts_tablet_manager.cc:1918] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:00:15.518150 4357 ts_tablet_manager.cc:1397] T 850114eb720f4c41b676635c961b6276 P a0f3243b474d4e25866098bc36f944c6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:00:15.518191 4396 log.cc:1196] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/f4213435c4dd400a8480f87c3efcb1d4
I20260808 23:00:15.518296 4357 raft_consensus.cc:348] T 850114eb720f4c41b676635c961b6276 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.518363 4357 raft_consensus.cc:374] T 850114eb720f4c41b676635c961b6276 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:15.518376 4356 tablet_bootstrap.cc:654] T 850114eb720f4c41b676635c961b6276 P 5de46e59381f438b9436e35be05cb093: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:15.518383 4357 raft_consensus.cc:729] T 850114eb720f4c41b676635c961b6276 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f3243b474d4e25866098bc36f944c6, State: Initialized, Role: FOLLOWER
I20260808 23:00:15.518426 4357 consensus_queue.cc:260] T 850114eb720f4c41b676635c961b6276 P a0f3243b474d4e25866098bc36f944c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.518507 4357 ts_tablet_manager.cc:1428] T 850114eb720f4c41b676635c961b6276 P a0f3243b474d4e25866098bc36f944c6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:15.518591 4397 tablet_replica.cc:330] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6: stopping tablet replica
I20260808 23:00:15.518676 4397 raft_consensus.cc:2229] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:00:15.518697 4365 raft_consensus.cc:482] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:00:15.518752 4365 raft_consensus.cc:504] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.518783 4397 raft_consensus.cc:2258] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:15.518997 4365 leader_election.cc:288] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0f3243b474d4e25866098bc36f944c6 (127.23.194.2:44439), 5de46e59381f438b9436e35be05cb093 (127.23.194.3:37529)
I20260808 23:00:15.519205 4397 ts_tablet_manager.cc:1905] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:15.518428 4396 ts_tablet_manager.cc:1939] T f4213435c4dd400a8480f87c3efcb1d4 P 5de46e59381f438b9436e35be05cb093: Deleting consensus metadata
I20260808 23:00:15.519802 3526 catalog_manager.cc:4841] TS 5de46e59381f438b9436e35be05cb093 (127.23.194.3:37529): tablet f4213435c4dd400a8480f87c3efcb1d4 (table test-table [id=907ba427f0884c07b8432f7a98ef2498]) successfully deleted
I20260808 23:00:15.520634 4397 ts_tablet_manager.cc:1918] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:00:15.520750 4356 tablet_bootstrap.cc:492] T 850114eb720f4c41b676635c961b6276 P 5de46e59381f438b9436e35be05cb093: No bootstrap required, opened a new log
I20260808 23:00:15.520762 4397 log.cc:1196] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/f4213435c4dd400a8480f87c3efcb1d4
I20260808 23:00:15.520794 4356 ts_tablet_manager.cc:1397] T 850114eb720f4c41b676635c961b6276 P 5de46e59381f438b9436e35be05cb093: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 23:00:15.520923 4356 raft_consensus.cc:348] T 850114eb720f4c41b676635c961b6276 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.520974 4356 raft_consensus.cc:374] T 850114eb720f4c41b676635c961b6276 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:15.520990 4356 raft_consensus.cc:729] T 850114eb720f4c41b676635c961b6276 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5de46e59381f438b9436e35be05cb093, State: Initialized, Role: FOLLOWER
I20260808 23:00:15.521003 4397 ts_tablet_manager.cc:1939] T f4213435c4dd400a8480f87c3efcb1d4 P a0f3243b474d4e25866098bc36f944c6: Deleting consensus metadata
I20260808 23:00:15.521050 4356 consensus_queue.cc:260] T 850114eb720f4c41b676635c961b6276 P 5de46e59381f438b9436e35be05cb093 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.521147 4356 ts_tablet_manager.cc:1428] T 850114eb720f4c41b676635c961b6276 P 5de46e59381f438b9436e35be05cb093: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:15.521405 3525 catalog_manager.cc:4841] TS a0f3243b474d4e25866098bc36f944c6 (127.23.194.2:44439): tablet f4213435c4dd400a8480f87c3efcb1d4 (table test-table [id=907ba427f0884c07b8432f7a98ef2498]) successfully deleted
I20260808 23:00:15.523254 4405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.519190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:44439 (local address 127.23.194.1:33191)
0808 23:00:15.519927 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:15.519937 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:15.519994 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:15.520155 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:15.520156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:15.520158 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:15.520222 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.520224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.521078 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.521079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.522300 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.522303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.523015 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.523017 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.523082 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.523087 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:00:15.523141 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 23:00:15.523176 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"thread_start_us":55,"threads_started":1}
I20260808 23:00:15.523257 4353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.519328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:33191 (local address 127.23.194.2:44439)
0808 23:00:15.519362 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.519364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.519997 (+ 633us) server_negotiation.cc:407] Connection header received
0808 23:00:15.520048 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.520049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.520060 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.520103 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:15.520307 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.520977 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.522382 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.522915 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.522993 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.523002 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.523153 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.523197 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 23:00:15.523707 4023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "850114eb720f4c41b676635c961b6276" candidate_uuid: "9c2ec16fd44742b788f43a95215c08e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0f3243b474d4e25866098bc36f944c6" is_pre_election: true
I20260808 23:00:15.523801 4023 raft_consensus.cc:2454] T 850114eb720f4c41b676635c961b6276 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2ec16fd44742b788f43a95215c08e7 in term 0.
I20260808 23:00:15.523977 3696 leader_election.cc:302] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2ec16fd44742b788f43a95215c08e7, a0f3243b474d4e25866098bc36f944c6; no voters:
I20260808 23:00:15.524071 4365 raft_consensus.cc:2781] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:00:15.524104 4365 raft_consensus.cc:482] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:00:15.524127 4365 raft_consensus.cc:3037] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:15.524458 4406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.519470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:37529 (local address 127.23.194.1:48345)
0808 23:00:15.520268 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:15.520277 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:15.520340 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:15.520485 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:15.520486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:15.520502 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:15.520560 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.520561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.522599 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.522600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.523473 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.523476 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.524202 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.524203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.524276 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.524280 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:00:15.524332 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 23:00:15.524369 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"mutex_wait_us":10,"thread_start_us":73,"threads_started":1}
I20260808 23:00:15.524705 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.519617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:48345 (local address 127.23.194.3:37529)
0808 23:00:15.519662 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.519664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.520320 (+ 656us) server_negotiation.cc:407] Connection header received
0808 23:00:15.520399 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.520400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.520408 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.520434 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:15.521930 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.522533 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.523550 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.524081 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.524150 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.524167 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.524611 (+ 444us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.524651 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 23:00:15.524832 4365 raft_consensus.cc:504] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.524921 4365 leader_election.cc:288] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [CANDIDATE]: Term 1 election: Requested vote from peers a0f3243b474d4e25866098bc36f944c6 (127.23.194.2:44439), 5de46e59381f438b9436e35be05cb093 (127.23.194.3:37529)
I20260808 23:00:15.525105 4023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "850114eb720f4c41b676635c961b6276" candidate_uuid: "9c2ec16fd44742b788f43a95215c08e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0f3243b474d4e25866098bc36f944c6"
I20260808 23:00:15.525116 4231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "850114eb720f4c41b676635c961b6276" candidate_uuid: "9c2ec16fd44742b788f43a95215c08e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de46e59381f438b9436e35be05cb093" is_pre_election: true
I20260808 23:00:15.525177 4023 raft_consensus.cc:3037] T 850114eb720f4c41b676635c961b6276 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:15.525195 4231 raft_consensus.cc:2454] T 850114eb720f4c41b676635c961b6276 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2ec16fd44742b788f43a95215c08e7 in term 0.
I20260808 23:00:15.525277 4230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "850114eb720f4c41b676635c961b6276" candidate_uuid: "9c2ec16fd44742b788f43a95215c08e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de46e59381f438b9436e35be05cb093"
I20260808 23:00:15.525319 4230 raft_consensus.cc:3037] T 850114eb720f4c41b676635c961b6276 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:15.525784 4023 raft_consensus.cc:2454] T 850114eb720f4c41b676635c961b6276 P a0f3243b474d4e25866098bc36f944c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2ec16fd44742b788f43a95215c08e7 in term 1.
I20260808 23:00:15.525941 3696 leader_election.cc:302] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2ec16fd44742b788f43a95215c08e7, a0f3243b474d4e25866098bc36f944c6; no voters:
I20260808 23:00:15.526024 4230 raft_consensus.cc:2454] T 850114eb720f4c41b676635c961b6276 P 5de46e59381f438b9436e35be05cb093 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2ec16fd44742b788f43a95215c08e7 in term 1.
I20260808 23:00:15.526032 4365 raft_consensus.cc:2781] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:15.526161 4365 raft_consensus.cc:686] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2ec16fd44742b788f43a95215c08e7, State: Running, Role: LEADER
I20260808 23:00:15.526219 4365 consensus_queue.cc:237] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.526789 3557 catalog_manager.cc:5495] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2ec16fd44742b788f43a95215c08e7 (127.23.194.1). New cstate: current_term: 1 leader_uuid: "9c2ec16fd44742b788f43a95215c08e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } health_report { overall_health: UNKNOWN } } }
Aug 08 23:00:15 dist-test-slave-f0tn krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230014, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.62@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM
I20260808 23:00:15.552973 3638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.546071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38700 (local address 127.23.194.62:33209)
0808 23:00:15.546118 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.546120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.546227 (+ 107us) server_negotiation.cc:407] Connection header received
0808 23:00:15.546271 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.546272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.546282 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.546348 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:15.546748 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.547513 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.548162 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.548459 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.548524 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.550725 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:15.550726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:15.550726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:15.550732 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:15.551933 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:15.552346 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:15.552347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:15.552347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:15.552394 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:15.552563 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:15.552564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:15.552564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:15.552653 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:15.552698 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.552849 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.552912 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 23:00:15.553083 4419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.545987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:33209 (local address 127.0.0.1:38700)
0808 23:00:15.546159 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:15.546170 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:15.546224 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:15.546351 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:15.546352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:15.546623 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:15.546679 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.546681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.547582 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.547583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.548091 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.548092 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.548518 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.548519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.548552 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.549043 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:15.549048 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:15.550629 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:15.552003 (+ 1374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:15.552011 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:15.552013 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:15.552281 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:15.552442 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:15.552443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:15.552443 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:15.552512 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:15.552704 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:15.552711 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:15.552763 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 23:00:15.552835 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 23:00:15.552982 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":39,"thread_start_us":90,"threads_started":1}
I20260808 23:00:15.555002 3557 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:38700:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 23:00:15.555231 3557 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:00:15.556510 3995 tablet_service.cc:1467] Processing CreateTablet for tablet 49b663af0c554c4eb6e7c0b4d6bf4ab7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ec140fd283a044b89020caf552e5fa9e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:00:15.556569 3765 tablet_service.cc:1467] Processing CreateTablet for tablet 49b663af0c554c4eb6e7c0b4d6bf4ab7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ec140fd283a044b89020caf552e5fa9e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:00:15.556661 3995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49b663af0c554c4eb6e7c0b4d6bf4ab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:15.556667 4202 tablet_service.cc:1467] Processing CreateTablet for tablet 49b663af0c554c4eb6e7c0b4d6bf4ab7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ec140fd283a044b89020caf552e5fa9e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:00:15.556716 3765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49b663af0c554c4eb6e7c0b4d6bf4ab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:15.556784 4202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49b663af0c554c4eb6e7c0b4d6bf4ab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:15.558171 4357 tablet_bootstrap.cc:492] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6: Bootstrap starting.
I20260808 23:00:15.558187 4356 tablet_bootstrap.cc:492] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 5de46e59381f438b9436e35be05cb093: Bootstrap starting.
I20260808 23:00:15.558562 4356 tablet_bootstrap.cc:654] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 5de46e59381f438b9436e35be05cb093: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:15.558575 4357 tablet_bootstrap.cc:654] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:15.558943 4354 tablet_bootstrap.cc:492] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 9c2ec16fd44742b788f43a95215c08e7: Bootstrap starting.
I20260808 23:00:15.559250 4357 tablet_bootstrap.cc:492] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6: No bootstrap required, opened a new log
I20260808 23:00:15.559303 4357 ts_tablet_manager.cc:1397] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 23:00:15.559317 4354 tablet_bootstrap.cc:654] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 9c2ec16fd44742b788f43a95215c08e7: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:15.559441 4357 raft_consensus.cc:348] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.559509 4357 raft_consensus.cc:374] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:15.559528 4357 raft_consensus.cc:729] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f3243b474d4e25866098bc36f944c6, State: Initialized, Role: FOLLOWER
I20260808 23:00:15.559569 4357 consensus_queue.cc:260] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.559650 4357 ts_tablet_manager.cc:1428] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:15.559748 4385 tablet_replica.cc:439] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } } }
I20260808 23:00:15.559821 4385 tablet_replica.cc:442] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:00:15.559895 4354 tablet_bootstrap.cc:492] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 9c2ec16fd44742b788f43a95215c08e7: No bootstrap required, opened a new log
I20260808 23:00:15.559934 4354 ts_tablet_manager.cc:1397] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 9c2ec16fd44742b788f43a95215c08e7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 23:00:15.560058 4354 raft_consensus.cc:348] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.560133 4354 raft_consensus.cc:374] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:15.560173 4354 raft_consensus.cc:729] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2ec16fd44742b788f43a95215c08e7, State: Initialized, Role: FOLLOWER
I20260808 23:00:15.560211 4354 consensus_queue.cc:260] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 9c2ec16fd44742b788f43a95215c08e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.560281 4356 tablet_bootstrap.cc:492] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 5de46e59381f438b9436e35be05cb093: No bootstrap required, opened a new log
I20260808 23:00:15.560297 4354 ts_tablet_manager.cc:1428] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 9c2ec16fd44742b788f43a95215c08e7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:15.560316 4356 ts_tablet_manager.cc:1397] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 5de46e59381f438b9436e35be05cb093: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:00:15.560451 4356 raft_consensus.cc:348] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.560523 4356 raft_consensus.cc:374] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:15.560564 4356 raft_consensus.cc:729] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5de46e59381f438b9436e35be05cb093, State: Initialized, Role: FOLLOWER
I20260808 23:00:15.560598 4356 consensus_queue.cc:260] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 5de46e59381f438b9436e35be05cb093 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.560694 4356 ts_tablet_manager.cc:1428] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 5de46e59381f438b9436e35be05cb093: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:15.560775 4366 tablet_replica.cc:439] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 5de46e59381f438b9436e35be05cb093: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } } }
I20260808 23:00:15.560819 4407 tablet_replica.cc:439] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 9c2ec16fd44742b788f43a95215c08e7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } } }
I20260808 23:00:15.560870 4366 tablet_replica.cc:442] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 5de46e59381f438b9436e35be05cb093: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:00:15.560894 4407 tablet_replica.cc:442] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 9c2ec16fd44742b788f43a95215c08e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:00:15.674965 4385 raft_consensus.cc:482] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:00:15.675056 4385 raft_consensus.cc:504] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.675230 4385 leader_election.cc:288] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2ec16fd44742b788f43a95215c08e7 (127.23.194.1:43967), 5de46e59381f438b9436e35be05cb093 (127.23.194.3:37529)
I20260808 23:00:15.675441 3792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49b663af0c554c4eb6e7c0b4d6bf4ab7" candidate_uuid: "a0f3243b474d4e25866098bc36f944c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2ec16fd44742b788f43a95215c08e7" is_pre_election: true
I20260808 23:00:15.675462 4230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49b663af0c554c4eb6e7c0b4d6bf4ab7" candidate_uuid: "a0f3243b474d4e25866098bc36f944c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de46e59381f438b9436e35be05cb093" is_pre_election: true
I20260808 23:00:15.675541 4230 raft_consensus.cc:2454] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0f3243b474d4e25866098bc36f944c6 in term 0.
I20260808 23:00:15.675541 3792 raft_consensus.cc:2454] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0f3243b474d4e25866098bc36f944c6 in term 0.
I20260808 23:00:15.675691 3911 leader_election.cc:302] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2ec16fd44742b788f43a95215c08e7, a0f3243b474d4e25866098bc36f944c6; no voters:
I20260808 23:00:15.675801 4385 raft_consensus.cc:2781] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:00:15.675838 4385 raft_consensus.cc:482] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:00:15.675853 4385 raft_consensus.cc:3037] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:15.676512 4385 raft_consensus.cc:504] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.676625 4385 leader_election.cc:288] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6 [CANDIDATE]: Term 1 election: Requested vote from peers 9c2ec16fd44742b788f43a95215c08e7 (127.23.194.1:43967), 5de46e59381f438b9436e35be05cb093 (127.23.194.3:37529)
I20260808 23:00:15.676789 3792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49b663af0c554c4eb6e7c0b4d6bf4ab7" candidate_uuid: "a0f3243b474d4e25866098bc36f944c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2ec16fd44742b788f43a95215c08e7"
I20260808 23:00:15.676805 4230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49b663af0c554c4eb6e7c0b4d6bf4ab7" candidate_uuid: "a0f3243b474d4e25866098bc36f944c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de46e59381f438b9436e35be05cb093"
I20260808 23:00:15.676872 3792 raft_consensus.cc:3037] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:15.676872 4230 raft_consensus.cc:3037] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:15.677678 3792 raft_consensus.cc:2454] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 9c2ec16fd44742b788f43a95215c08e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0f3243b474d4e25866098bc36f944c6 in term 1.
I20260808 23:00:15.677673 4230 raft_consensus.cc:2454] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 5de46e59381f438b9436e35be05cb093 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0f3243b474d4e25866098bc36f944c6 in term 1.
I20260808 23:00:15.677834 3911 leader_election.cc:302] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2ec16fd44742b788f43a95215c08e7, a0f3243b474d4e25866098bc36f944c6; no voters:
I20260808 23:00:15.677948 4385 raft_consensus.cc:2781] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:15.678013 4385 raft_consensus.cc:686] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6 [term 1 LEADER]: Becoming Leader. State: Replica: a0f3243b474d4e25866098bc36f944c6, State: Running, Role: LEADER
I20260808 23:00:15.678074 4385 consensus_queue.cc:237] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.678329 4371 tablet_replica.cc:439] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6: TxnStatusTablet state changed. Reason: New leader a0f3243b474d4e25866098bc36f944c6. Latest consensus state: current_term: 1 leader_uuid: "a0f3243b474d4e25866098bc36f944c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } } }
I20260808 23:00:15.678419 4371 tablet_replica.cc:442] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:00:15.678561 4430 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 23:00:15.678683 4430 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 23:00:15.678658 3559 catalog_manager.cc:5495] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0f3243b474d4e25866098bc36f944c6 (127.23.194.2). New cstate: current_term: 1 leader_uuid: "a0f3243b474d4e25866098bc36f944c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } health_report { overall_health: UNKNOWN } } }
I20260808 23:00:15.705488 4353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.703290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46030 (local address 127.23.194.2:44439)
0808 23:00:15.703337 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.703339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.703348 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:00:15.703424 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.703425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.703434 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.703480 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:00:15.703582 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.704051 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.704821 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.705017 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.705072 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.705234 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:00:15.705271 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:00:15.705314 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.705393 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.705418 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 23:00:15.705513 4419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.703205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:44439 (local address 127.0.0.1:46030)
0808 23:00:15.703302 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:15.703316 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:15.703365 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:15.703485 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:15.703486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:15.703487 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:00:15.703536 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.703539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.704126 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.704127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.704745 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.704747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.705079 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.705081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.705146 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.705165 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:00:15.705324 (+ 159us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:00:15.705332 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 23:00:15.705371 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 23:00:15.705446 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 23:00:15.706458 4230 raft_consensus.cc:1261] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 5de46e59381f438b9436e35be05cb093 [term 1 FOLLOWER]: Refusing update from remote peer a0f3243b474d4e25866098bc36f944c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:00:15.706697 3792 raft_consensus.cc:1261] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 9c2ec16fd44742b788f43a95215c08e7 [term 1 FOLLOWER]: Refusing update from remote peer a0f3243b474d4e25866098bc36f944c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:00:15.706669 4385 consensus_queue.cc:1035] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:15.706866 4385 consensus_queue.cc:1035] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:15.707206 4366 tablet_replica.cc:439] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 5de46e59381f438b9436e35be05cb093: TxnStatusTablet state changed. Reason: New leader a0f3243b474d4e25866098bc36f944c6. Latest consensus state: current_term: 1 leader_uuid: "a0f3243b474d4e25866098bc36f944c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } } }
I20260808 23:00:15.707298 4366 tablet_replica.cc:442] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 5de46e59381f438b9436e35be05cb093: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:00:15.707424 4407 tablet_replica.cc:439] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 9c2ec16fd44742b788f43a95215c08e7: TxnStatusTablet state changed. Reason: New leader a0f3243b474d4e25866098bc36f944c6. Latest consensus state: current_term: 1 leader_uuid: "a0f3243b474d4e25866098bc36f944c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } } }
I20260808 23:00:15.707512 4407 tablet_replica.cc:442] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 9c2ec16fd44742b788f43a95215c08e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:00:15.708087 4371 tablet_replica.cc:439] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0f3243b474d4e25866098bc36f944c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } } }
I20260808 23:00:15.708169 4371 tablet_replica.cc:442] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:00:15.708308 4379 mvcc.cc:205] Tried to move back new op lower bound from 7316398144332111872 to 7316398144217829376. Current Snapshot: MvccSnapshot[applied={T|T < 7316398144332111872}]
I20260808 23:00:15.708452 4366 tablet_replica.cc:439] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 5de46e59381f438b9436e35be05cb093: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0f3243b474d4e25866098bc36f944c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } } }
I20260808 23:00:15.708534 4366 tablet_replica.cc:442] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 5de46e59381f438b9436e35be05cb093: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:00:15.709782 4368 tablet_replica.cc:439] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0f3243b474d4e25866098bc36f944c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } } }
I20260808 23:00:15.709867 4368 tablet_replica.cc:442] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P a0f3243b474d4e25866098bc36f944c6: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:00:15.709978 4407 tablet_replica.cc:439] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 9c2ec16fd44742b788f43a95215c08e7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0f3243b474d4e25866098bc36f944c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } } }
I20260808 23:00:15.710049 4407 tablet_replica.cc:442] T 49b663af0c554c4eb6e7c0b4d6bf4ab7 P 9c2ec16fd44742b788f43a95215c08e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:00:15.712764 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.710071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55226 (local address 127.23.194.1:43967)
0808 23:00:15.710107 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.710109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.710256 (+ 147us) server_negotiation.cc:407] Connection header received
0808 23:00:15.710318 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.710319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.710339 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.710409 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:00:15.710561 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.711059 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.712072 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.712242 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.712284 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.712429 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:00:15.712567 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:00:15.712620 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.712676 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.712696 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 23:00:15.717730 4353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.715179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:47845 (local address 127.23.194.2:44439)
0808 23:00:15.715221 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.715223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.715351 (+ 128us) server_negotiation.cc:407] Connection header received
0808 23:00:15.715411 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.715412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.715420 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.715463 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:00:15.715595 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.716012 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.717089 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.717269 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.717319 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.717532 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:00:15.717563 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:00:15.717614 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.717669 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.717686 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 23:00:15.717952 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.715119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:44439 (local address 127.23.194.1:47845)
0808 23:00:15.715293 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:15.715303 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:15.715360 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:15.715466 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:15.715467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:15.715468 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:00:15.715534 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.715535 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.716071 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.716072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.717019 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.717022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.717333 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.717334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.717442 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.717458 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:00:15.717620 (+ 162us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:00:15.717621 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 23:00:15.717669 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 23:00:15.717714 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":85,"threads_started":1}
I20260808 23:00:15.720007 4230 raft_consensus.cc:1261] T 850114eb720f4c41b676635c961b6276 P 5de46e59381f438b9436e35be05cb093 [term 1 FOLLOWER]: Refusing update from remote peer 9c2ec16fd44742b788f43a95215c08e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:00:15.720217 4365 consensus_queue.cc:1035] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:15.720312 4023 raft_consensus.cc:1261] T 850114eb720f4c41b676635c961b6276 P a0f3243b474d4e25866098bc36f944c6 [term 1 FOLLOWER]: Refusing update from remote peer 9c2ec16fd44742b788f43a95215c08e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:00:15.720579 4365 consensus_queue.cc:1035] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:15.728399 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.725808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:43137 (local address 127.23.194.1:43967)
0808 23:00:15.725893 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.725895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.725902 (+ 7us) server_negotiation.cc:407] Connection header received
0808 23:00:15.725969 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.725970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.725978 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.726030 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:00:15.726170 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.726644 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.727703 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.727988 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.728024 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.728223 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:00:15.728250 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:00:15.728283 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.728338 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.728355 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 23:00:15.728621 4443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.725619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:43967 (local address 127.23.194.2:43137)
0808 23:00:15.725848 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:15.725858 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:15.725904 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:15.726070 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:15.726071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:15.726073 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:00:15.726115 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.726116 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.726716 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.726717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.727589 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.727592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.728082 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.728083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.728138 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.728153 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:00:15.728286 (+ 133us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:00:15.728287 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 23:00:15.728334 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 23:00:15.728375 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":93,"threads_started":1}
I20260808 23:00:15.738022 3557 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:00:15.738126 3557 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:00:15.738754 3557 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7: Sending DeleteTablet for 3 replicas of tablet 850114eb720f4c41b676635c961b6276
I20260808 23:00:15.739087 3995 tablet_service.cc:1514] Processing DeleteTablet for tablet 850114eb720f4c41b676635c961b6276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:15 UTC) from {username='oryx', principal='oryx/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:46010
I20260808 23:00:15.739089 4202 tablet_service.cc:1514] Processing DeleteTablet for tablet 850114eb720f4c41b676635c961b6276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:15 UTC) from {username='oryx', principal='oryx/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:43354
I20260808 23:00:15.739184 4397 tablet_replica.cc:330] T 850114eb720f4c41b676635c961b6276 P a0f3243b474d4e25866098bc36f944c6: stopping tablet replica
I20260808 23:00:15.739184 4396 tablet_replica.cc:330] T 850114eb720f4c41b676635c961b6276 P 5de46e59381f438b9436e35be05cb093: stopping tablet replica
I20260808 23:00:15.739230 4397 raft_consensus.cc:2229] T 850114eb720f4c41b676635c961b6276 P a0f3243b474d4e25866098bc36f944c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:00:15.739235 4396 raft_consensus.cc:2229] T 850114eb720f4c41b676635c961b6276 P 5de46e59381f438b9436e35be05cb093 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:00:15.739264 4397 raft_consensus.cc:2258] T 850114eb720f4c41b676635c961b6276 P a0f3243b474d4e25866098bc36f944c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:15.739269 4396 raft_consensus.cc:2258] T 850114eb720f4c41b676635c961b6276 P 5de46e59381f438b9436e35be05cb093 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:15.739564 4397 ts_tablet_manager.cc:1905] T 850114eb720f4c41b676635c961b6276 P a0f3243b474d4e25866098bc36f944c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:15.739636 3765 tablet_service.cc:1514] Processing DeleteTablet for tablet 850114eb720f4c41b676635c961b6276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:15 UTC) from {username='oryx', principal='oryx/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:55212
I20260808 23:00:15.739645 4396 ts_tablet_manager.cc:1905] T 850114eb720f4c41b676635c961b6276 P 5de46e59381f438b9436e35be05cb093: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:15.739730 4394 tablet_replica.cc:330] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7: stopping tablet replica
I20260808 23:00:15.739794 4394 raft_consensus.cc:2229] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:00:15.739886 4394 raft_consensus.cc:2258] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:15.740211 4394 ts_tablet_manager.cc:1905] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:15.741539 4396 ts_tablet_manager.cc:1918] T 850114eb720f4c41b676635c961b6276 P 5de46e59381f438b9436e35be05cb093: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:00:15.741602 4396 log.cc:1196] T 850114eb720f4c41b676635c961b6276 P 5de46e59381f438b9436e35be05cb093: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/850114eb720f4c41b676635c961b6276
I20260808 23:00:15.741673 4397 ts_tablet_manager.cc:1918] T 850114eb720f4c41b676635c961b6276 P a0f3243b474d4e25866098bc36f944c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:00:15.741690 4394 ts_tablet_manager.cc:1918] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:00:15.741740 4397 log.cc:1196] T 850114eb720f4c41b676635c961b6276 P a0f3243b474d4e25866098bc36f944c6: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/850114eb720f4c41b676635c961b6276
I20260808 23:00:15.741745 4394 log.cc:1196] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/wal/wals/850114eb720f4c41b676635c961b6276
I20260808 23:00:15.741834 4396 ts_tablet_manager.cc:1939] T 850114eb720f4c41b676635c961b6276 P 5de46e59381f438b9436e35be05cb093: Deleting consensus metadata
I20260808 23:00:15.741946 4394 ts_tablet_manager.cc:1939] T 850114eb720f4c41b676635c961b6276 P 9c2ec16fd44742b788f43a95215c08e7: Deleting consensus metadata
I20260808 23:00:15.741946 4397 ts_tablet_manager.cc:1939] T 850114eb720f4c41b676635c961b6276 P a0f3243b474d4e25866098bc36f944c6: Deleting consensus metadata
I20260808 23:00:15.742269 3525 catalog_manager.cc:4841] TS a0f3243b474d4e25866098bc36f944c6 (127.23.194.2:44439): tablet 850114eb720f4c41b676635c961b6276 (table test-table [id=5cce6cd22c9a4c88a0a76138bf279dcb]) successfully deleted
I20260808 23:00:15.742362 3527 catalog_manager.cc:4841] TS 9c2ec16fd44742b788f43a95215c08e7 (127.23.194.1:43967): tablet 850114eb720f4c41b676635c961b6276 (table test-table [id=5cce6cd22c9a4c88a0a76138bf279dcb]) successfully deleted
I20260808 23:00:15.742815 3526 catalog_manager.cc:4841] TS 5de46e59381f438b9436e35be05cb093 (127.23.194.3:37529): tablet 850114eb720f4c41b676635c961b6276 (table test-table [id=5cce6cd22c9a4c88a0a76138bf279dcb]) successfully deleted
Aug 08 23:00:15 dist-test-slave-f0tn krb5kdc[3475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 23:00:15 dist-test-slave-f0tn krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:15 dist-test-slave-f0tn krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 23:00:15.769902 3638 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:15.763161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38708 (local address 127.23.194.62:33209)
0808 23:00:15.763208 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.763211 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.763365 (+ 154us) server_negotiation.cc:407] Connection header received
0808 23:00:15.763422 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.763423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.763437 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.763501 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:15.764179 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.764960 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.766123 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.766422 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.766499 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.769836 (+ 3337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38708: BlockingRecv error: recv got EOF from 127.0.0.1:38708 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 08 23:00:15 dist-test-slave-f0tn krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM
I20260808 23:00:15.778836 3638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.770641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38712 (local address 127.23.194.62:33209)
0808 23:00:15.770690 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.770692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.770916 (+ 224us) server_negotiation.cc:407] Connection header received
0808 23:00:15.770963 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.770964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.770974 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.771034 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:15.771572 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.772293 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.773296 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.773625 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.773714 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.776167 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:15.776168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:15.776169 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:15.776176 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:15.777734 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:15.778111 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:15.778112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:15.778112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:15.778182 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:15.778357 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:15.778357 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:15.778358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:15.778482 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:15.778573 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.778671 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.778745 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 23:00:15.780402 3557 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:00:15.780578 3557 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:00:15.782044 3995 tablet_service.cc:1467] Processing CreateTablet for tablet dadd728f4f27473883ec4f9d25621611 (DEFAULT_TABLE table=test-table [id=df4ae74ac6c74351899537762a5591fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:15.782050 4202 tablet_service.cc:1467] Processing CreateTablet for tablet dadd728f4f27473883ec4f9d25621611 (DEFAULT_TABLE table=test-table [id=df4ae74ac6c74351899537762a5591fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:15.782186 4202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dadd728f4f27473883ec4f9d25621611. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:15.782196 3995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dadd728f4f27473883ec4f9d25621611. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:15.783511 4356 tablet_bootstrap.cc:492] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093: Bootstrap starting.
I20260808 23:00:15.783557 3765 tablet_service.cc:1467] Processing CreateTablet for tablet dadd728f4f27473883ec4f9d25621611 (DEFAULT_TABLE table=test-table [id=df4ae74ac6c74351899537762a5591fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:15.783668 3765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dadd728f4f27473883ec4f9d25621611. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:15.783836 4356 tablet_bootstrap.cc:654] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:15.784395 4356 tablet_bootstrap.cc:492] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093: No bootstrap required, opened a new log
I20260808 23:00:15.784431 4356 ts_tablet_manager.cc:1397] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 23:00:15.784687 4356 raft_consensus.cc:348] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.784736 4356 raft_consensus.cc:374] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:15.784749 4356 raft_consensus.cc:729] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5de46e59381f438b9436e35be05cb093, State: Initialized, Role: FOLLOWER
I20260808 23:00:15.784787 4356 consensus_queue.cc:260] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.784870 4356 ts_tablet_manager.cc:1428] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:15.784880 4354 tablet_bootstrap.cc:492] T dadd728f4f27473883ec4f9d25621611 P 9c2ec16fd44742b788f43a95215c08e7: Bootstrap starting.
I20260808 23:00:15.784912 4357 tablet_bootstrap.cc:492] T dadd728f4f27473883ec4f9d25621611 P a0f3243b474d4e25866098bc36f944c6: Bootstrap starting.
I20260808 23:00:15.785240 4357 tablet_bootstrap.cc:654] T dadd728f4f27473883ec4f9d25621611 P a0f3243b474d4e25866098bc36f944c6: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:15.785312 4354 tablet_bootstrap.cc:654] T dadd728f4f27473883ec4f9d25621611 P 9c2ec16fd44742b788f43a95215c08e7: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:15.785985 4354 tablet_bootstrap.cc:492] T dadd728f4f27473883ec4f9d25621611 P 9c2ec16fd44742b788f43a95215c08e7: No bootstrap required, opened a new log
I20260808 23:00:15.786037 4354 ts_tablet_manager.cc:1397] T dadd728f4f27473883ec4f9d25621611 P 9c2ec16fd44742b788f43a95215c08e7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 23:00:15.786190 4354 raft_consensus.cc:348] T dadd728f4f27473883ec4f9d25621611 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.786259 4354 raft_consensus.cc:374] T dadd728f4f27473883ec4f9d25621611 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:15.786278 4354 raft_consensus.cc:729] T dadd728f4f27473883ec4f9d25621611 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2ec16fd44742b788f43a95215c08e7, State: Initialized, Role: FOLLOWER
I20260808 23:00:15.786312 4354 consensus_queue.cc:260] T dadd728f4f27473883ec4f9d25621611 P 9c2ec16fd44742b788f43a95215c08e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.786388 4354 ts_tablet_manager.cc:1428] T dadd728f4f27473883ec4f9d25621611 P 9c2ec16fd44742b788f43a95215c08e7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:15.787249 4357 tablet_bootstrap.cc:492] T dadd728f4f27473883ec4f9d25621611 P a0f3243b474d4e25866098bc36f944c6: No bootstrap required, opened a new log
I20260808 23:00:15.787307 4357 ts_tablet_manager.cc:1397] T dadd728f4f27473883ec4f9d25621611 P a0f3243b474d4e25866098bc36f944c6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:00:15.787402 4357 raft_consensus.cc:348] T dadd728f4f27473883ec4f9d25621611 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.787472 4357 raft_consensus.cc:374] T dadd728f4f27473883ec4f9d25621611 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:15.787490 4357 raft_consensus.cc:729] T dadd728f4f27473883ec4f9d25621611 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f3243b474d4e25866098bc36f944c6, State: Initialized, Role: FOLLOWER
I20260808 23:00:15.787532 4357 consensus_queue.cc:260] T dadd728f4f27473883ec4f9d25621611 P a0f3243b474d4e25866098bc36f944c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.787621 4357 ts_tablet_manager.cc:1428] T dadd728f4f27473883ec4f9d25621611 P a0f3243b474d4e25866098bc36f944c6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:15.983516 4366 raft_consensus.cc:482] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:00:15.983618 4366 raft_consensus.cc:504] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.983954 4366 leader_election.cc:288] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2ec16fd44742b788f43a95215c08e7 (127.23.194.1:43967), a0f3243b474d4e25866098bc36f944c6 (127.23.194.2:44439)
I20260808 23:00:15.986815 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.984200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:45273 (local address 127.23.194.1:43967)
0808 23:00:15.984261 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.984263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.984349 (+ 86us) server_negotiation.cc:407] Connection header received
0808 23:00:15.984391 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.984392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.984407 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.984456 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:15.984636 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.985110 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.986150 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.986482 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.986541 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.986550 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.986715 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.986749 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 23:00:15.986814 4353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.984255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:40465 (local address 127.23.194.2:44439)
0808 23:00:15.984306 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:15.984308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:15.984317 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:00:15.984394 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:15.984395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:15.984406 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:15.984453 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:15.984614 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.985111 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.986139 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.986487 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.986551 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.986559 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:15.986707 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 23:00:15.986744 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 23:00:15.986851 4475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.984119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:44439 (local address 127.23.194.3:40465)
0808 23:00:15.984270 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:15.984280 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:15.984330 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:15.984466 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:15.984467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:15.984469 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:15.984558 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.984559 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.985202 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.985203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.986047 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.986050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.986577 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.986578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.986634 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.986645 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 23:00:15.986688 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 23:00:15.986728 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":18,"thread_start_us":78,"threads_started":1}
I20260808 23:00:15.986855 4476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:15.984119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:43967 (local address 127.23.194.3:45273)
0808 23:00:15.984265 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:15.984276 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:15.984342 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:15.984468 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:15.984469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:15.984471 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:15.984571 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.984573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.985208 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.985209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.986056 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:15.986058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:15.986572 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:15.986574 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:15.986646 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:15.986650 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:00:15.986699 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 23:00:15.986739 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":64,"thread_start_us":81,"threads_started":1}
I20260808 23:00:15.987140 3792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dadd728f4f27473883ec4f9d25621611" candidate_uuid: "5de46e59381f438b9436e35be05cb093" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2ec16fd44742b788f43a95215c08e7" is_pre_election: true
I20260808 23:00:15.987246 3792 raft_consensus.cc:2454] T dadd728f4f27473883ec4f9d25621611 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5de46e59381f438b9436e35be05cb093 in term 0.
I20260808 23:00:15.987241 4023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dadd728f4f27473883ec4f9d25621611" candidate_uuid: "5de46e59381f438b9436e35be05cb093" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0f3243b474d4e25866098bc36f944c6" is_pre_election: true
I20260808 23:00:15.987334 4023 raft_consensus.cc:2454] T dadd728f4f27473883ec4f9d25621611 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5de46e59381f438b9436e35be05cb093 in term 0.
I20260808 23:00:15.987413 4131 leader_election.cc:302] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5de46e59381f438b9436e35be05cb093, 9c2ec16fd44742b788f43a95215c08e7; no voters:
I20260808 23:00:15.987543 4366 raft_consensus.cc:2781] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:00:15.987584 4366 raft_consensus.cc:482] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:00:15.987598 4366 raft_consensus.cc:3037] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:15.988245 4366 raft_consensus.cc:504] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.988358 4366 leader_election.cc:288] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [CANDIDATE]: Term 1 election: Requested vote from peers 9c2ec16fd44742b788f43a95215c08e7 (127.23.194.1:43967), a0f3243b474d4e25866098bc36f944c6 (127.23.194.2:44439)
I20260808 23:00:15.988521 3792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dadd728f4f27473883ec4f9d25621611" candidate_uuid: "5de46e59381f438b9436e35be05cb093" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2ec16fd44742b788f43a95215c08e7"
I20260808 23:00:15.988502 4023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dadd728f4f27473883ec4f9d25621611" candidate_uuid: "5de46e59381f438b9436e35be05cb093" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0f3243b474d4e25866098bc36f944c6"
I20260808 23:00:15.988587 4023 raft_consensus.cc:3037] T dadd728f4f27473883ec4f9d25621611 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:15.988586 3792 raft_consensus.cc:3037] T dadd728f4f27473883ec4f9d25621611 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:15.989351 3792 raft_consensus.cc:2454] T dadd728f4f27473883ec4f9d25621611 P 9c2ec16fd44742b788f43a95215c08e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5de46e59381f438b9436e35be05cb093 in term 1.
I20260808 23:00:15.989355 4023 raft_consensus.cc:2454] T dadd728f4f27473883ec4f9d25621611 P a0f3243b474d4e25866098bc36f944c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5de46e59381f438b9436e35be05cb093 in term 1.
I20260808 23:00:15.989526 4129 leader_election.cc:302] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5de46e59381f438b9436e35be05cb093, a0f3243b474d4e25866098bc36f944c6; no voters:
I20260808 23:00:15.989606 4366 raft_consensus.cc:2781] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:15.989717 4366 raft_consensus.cc:686] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [term 1 LEADER]: Becoming Leader. State: Replica: 5de46e59381f438b9436e35be05cb093, State: Running, Role: LEADER
I20260808 23:00:15.989789 4366 consensus_queue.cc:237] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:15.990432 3559 catalog_manager.cc:5495] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5de46e59381f438b9436e35be05cb093 (127.23.194.3). New cstate: current_term: 1 leader_uuid: "5de46e59381f438b9436e35be05cb093" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } health_report { overall_health: HEALTHY } } }
I20260808 23:00:16.014770 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:16.012743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43368 (local address 127.23.194.3:37529)
0808 23:00:16.012798 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:16.012800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:16.012824 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:00:16.012859 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:16.012860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:16.012882 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:16.012935 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:00:16.013040 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:16.013543 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:16.014221 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:16.014389 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:16.014431 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:16.014535 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:00:16.014632 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:00:16.014683 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:16.014697 (+ 14us) server_negotiation.cc:299] Negotiation successful
0808 23:00:16.014715 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 23:00:16.015668 3792 raft_consensus.cc:1261] T dadd728f4f27473883ec4f9d25621611 P 9c2ec16fd44742b788f43a95215c08e7 [term 1 FOLLOWER]: Refusing update from remote peer 5de46e59381f438b9436e35be05cb093: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:00:16.015695 4023 raft_consensus.cc:1261] T dadd728f4f27473883ec4f9d25621611 P a0f3243b474d4e25866098bc36f944c6 [term 1 FOLLOWER]: Refusing update from remote peer 5de46e59381f438b9436e35be05cb093: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:00:16.015928 4366 consensus_queue.cc:1035] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:16.016073 4478 consensus_queue.cc:1035] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:16.019529 3557 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:00:16.019613 3557 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:00:16.020190 3557 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7: Sending DeleteTablet for 3 replicas of tablet dadd728f4f27473883ec4f9d25621611
I20260808 23:00:16.020423 3995 tablet_service.cc:1514] Processing DeleteTablet for tablet dadd728f4f27473883ec4f9d25621611 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:16 UTC) from {username='oryx', principal='oryx/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:46010
I20260808 23:00:16.020433 3765 tablet_service.cc:1514] Processing DeleteTablet for tablet dadd728f4f27473883ec4f9d25621611 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:16 UTC) from {username='oryx', principal='oryx/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:55212
I20260808 23:00:16.020445 4202 tablet_service.cc:1514] Processing DeleteTablet for tablet dadd728f4f27473883ec4f9d25621611 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:16 UTC) from {username='oryx', principal='oryx/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:43354
I20260808 23:00:16.020511 4397 tablet_replica.cc:330] T dadd728f4f27473883ec4f9d25621611 P a0f3243b474d4e25866098bc36f944c6: stopping tablet replica
I20260808 23:00:16.020522 4396 tablet_replica.cc:330] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093: stopping tablet replica
I20260808 23:00:16.020556 4397 raft_consensus.cc:2229] T dadd728f4f27473883ec4f9d25621611 P a0f3243b474d4e25866098bc36f944c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:00:16.020567 4396 raft_consensus.cc:2229] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:00:16.020596 4397 raft_consensus.cc:2258] T dadd728f4f27473883ec4f9d25621611 P a0f3243b474d4e25866098bc36f944c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:16.020630 4396 raft_consensus.cc:2258] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:16.020705 4394 tablet_replica.cc:330] T dadd728f4f27473883ec4f9d25621611 P 9c2ec16fd44742b788f43a95215c08e7: stopping tablet replica
I20260808 23:00:16.020766 4394 raft_consensus.cc:2229] T dadd728f4f27473883ec4f9d25621611 P 9c2ec16fd44742b788f43a95215c08e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:00:16.020802 4394 raft_consensus.cc:2258] T dadd728f4f27473883ec4f9d25621611 P 9c2ec16fd44742b788f43a95215c08e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:16.020959 4397 ts_tablet_manager.cc:1905] T dadd728f4f27473883ec4f9d25621611 P a0f3243b474d4e25866098bc36f944c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:16.021152 3557 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:00:16.021291 3557 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:00:16.021644 4396 ts_tablet_manager.cc:1905] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:16.022612 4397 ts_tablet_manager.cc:1918] T dadd728f4f27473883ec4f9d25621611 P a0f3243b474d4e25866098bc36f944c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:00:16.022673 4397 log.cc:1196] T dadd728f4f27473883ec4f9d25621611 P a0f3243b474d4e25866098bc36f944c6: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/dadd728f4f27473883ec4f9d25621611
I20260808 23:00:16.022935 4396 ts_tablet_manager.cc:1918] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:00:16.022970 4397 ts_tablet_manager.cc:1939] T dadd728f4f27473883ec4f9d25621611 P a0f3243b474d4e25866098bc36f944c6: Deleting consensus metadata
I20260808 23:00:16.022993 4396 log.cc:1196] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/dadd728f4f27473883ec4f9d25621611
I20260808 23:00:16.023161 4396 ts_tablet_manager.cc:1939] T dadd728f4f27473883ec4f9d25621611 P 5de46e59381f438b9436e35be05cb093: Deleting consensus metadata
I20260808 23:00:16.023478 3525 catalog_manager.cc:4841] TS a0f3243b474d4e25866098bc36f944c6 (127.23.194.2:44439): tablet dadd728f4f27473883ec4f9d25621611 (table test-table [id=df4ae74ac6c74351899537762a5591fd]) successfully deleted
I20260808 23:00:16.023623 3526 catalog_manager.cc:4841] TS 5de46e59381f438b9436e35be05cb093 (127.23.194.3:37529): tablet dadd728f4f27473883ec4f9d25621611 (table test-table [id=df4ae74ac6c74351899537762a5591fd]) successfully deleted
I20260808 23:00:16.024044 4394 ts_tablet_manager.cc:1905] T dadd728f4f27473883ec4f9d25621611 P 9c2ec16fd44742b788f43a95215c08e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:16.024102 3765 tablet_service.cc:1467] Processing CreateTablet for tablet bf4f0d6137d2413382a2530b21f66f82 (DEFAULT_TABLE table=test-table [id=d43e57bb05a54facbdb6ce049758b20b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:16.024204 3765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf4f0d6137d2413382a2530b21f66f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:16.025422 4394 ts_tablet_manager.cc:1918] T dadd728f4f27473883ec4f9d25621611 P 9c2ec16fd44742b788f43a95215c08e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:00:16.025511 4394 log.cc:1196] T dadd728f4f27473883ec4f9d25621611 P 9c2ec16fd44742b788f43a95215c08e7: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/wal/wals/dadd728f4f27473883ec4f9d25621611
I20260808 23:00:16.025714 4394 ts_tablet_manager.cc:1939] T dadd728f4f27473883ec4f9d25621611 P 9c2ec16fd44742b788f43a95215c08e7: Deleting consensus metadata
I20260808 23:00:16.025983 4202 tablet_service.cc:1467] Processing CreateTablet for tablet bf4f0d6137d2413382a2530b21f66f82 (DEFAULT_TABLE table=test-table [id=d43e57bb05a54facbdb6ce049758b20b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:16.026023 3527 catalog_manager.cc:4841] TS 9c2ec16fd44742b788f43a95215c08e7 (127.23.194.1:43967): tablet dadd728f4f27473883ec4f9d25621611 (table test-table [id=df4ae74ac6c74351899537762a5591fd]) successfully deleted
I20260808 23:00:16.026113 4202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf4f0d6137d2413382a2530b21f66f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:16.025372 4354 tablet_bootstrap.cc:492] T bf4f0d6137d2413382a2530b21f66f82 P 9c2ec16fd44742b788f43a95215c08e7: Bootstrap starting.
I20260808 23:00:16.026934 4356 tablet_bootstrap.cc:492] T bf4f0d6137d2413382a2530b21f66f82 P 5de46e59381f438b9436e35be05cb093: Bootstrap starting.
I20260808 23:00:16.027184 4354 tablet_bootstrap.cc:654] T bf4f0d6137d2413382a2530b21f66f82 P 9c2ec16fd44742b788f43a95215c08e7: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:16.027366 4356 tablet_bootstrap.cc:654] T bf4f0d6137d2413382a2530b21f66f82 P 5de46e59381f438b9436e35be05cb093: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:16.027513 3995 tablet_service.cc:1467] Processing CreateTablet for tablet bf4f0d6137d2413382a2530b21f66f82 (DEFAULT_TABLE table=test-table [id=d43e57bb05a54facbdb6ce049758b20b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:16.027621 3995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf4f0d6137d2413382a2530b21f66f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:16.028556 4357 tablet_bootstrap.cc:492] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6: Bootstrap starting.
I20260808 23:00:16.028684 4354 tablet_bootstrap.cc:492] T bf4f0d6137d2413382a2530b21f66f82 P 9c2ec16fd44742b788f43a95215c08e7: No bootstrap required, opened a new log
I20260808 23:00:16.028743 4354 ts_tablet_manager.cc:1397] T bf4f0d6137d2413382a2530b21f66f82 P 9c2ec16fd44742b788f43a95215c08e7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 23:00:16.028837 4354 raft_consensus.cc:348] T bf4f0d6137d2413382a2530b21f66f82 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:16.028884 4354 raft_consensus.cc:374] T bf4f0d6137d2413382a2530b21f66f82 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:16.028896 4354 raft_consensus.cc:729] T bf4f0d6137d2413382a2530b21f66f82 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2ec16fd44742b788f43a95215c08e7, State: Initialized, Role: FOLLOWER
I20260808 23:00:16.028931 4354 consensus_queue.cc:260] T bf4f0d6137d2413382a2530b21f66f82 P 9c2ec16fd44742b788f43a95215c08e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:16.028962 4357 tablet_bootstrap.cc:654] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:16.029016 4354 ts_tablet_manager.cc:1428] T bf4f0d6137d2413382a2530b21f66f82 P 9c2ec16fd44742b788f43a95215c08e7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:16.029501 4356 tablet_bootstrap.cc:492] T bf4f0d6137d2413382a2530b21f66f82 P 5de46e59381f438b9436e35be05cb093: No bootstrap required, opened a new log
I20260808 23:00:16.029553 4356 ts_tablet_manager.cc:1397] T bf4f0d6137d2413382a2530b21f66f82 P 5de46e59381f438b9436e35be05cb093: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 23:00:16.029676 4356 raft_consensus.cc:348] T bf4f0d6137d2413382a2530b21f66f82 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:16.029754 4356 raft_consensus.cc:374] T bf4f0d6137d2413382a2530b21f66f82 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:16.029773 4356 raft_consensus.cc:729] T bf4f0d6137d2413382a2530b21f66f82 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5de46e59381f438b9436e35be05cb093, State: Initialized, Role: FOLLOWER
I20260808 23:00:16.029816 4356 consensus_queue.cc:260] T bf4f0d6137d2413382a2530b21f66f82 P 5de46e59381f438b9436e35be05cb093 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:16.029899 4356 ts_tablet_manager.cc:1428] T bf4f0d6137d2413382a2530b21f66f82 P 5de46e59381f438b9436e35be05cb093: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:16.030189 4357 tablet_bootstrap.cc:492] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6: No bootstrap required, opened a new log
I20260808 23:00:16.030239 4357 ts_tablet_manager.cc:1397] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:00:16.030337 4357 raft_consensus.cc:348] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:16.030405 4357 raft_consensus.cc:374] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:16.030424 4357 raft_consensus.cc:729] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f3243b474d4e25866098bc36f944c6, State: Initialized, Role: FOLLOWER
I20260808 23:00:16.030484 4357 consensus_queue.cc:260] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:16.030570 4357 ts_tablet_manager.cc:1428] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:16.251272 4371 raft_consensus.cc:482] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:00:16.251371 4371 raft_consensus.cc:504] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:16.251518 4371 leader_election.cc:288] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2ec16fd44742b788f43a95215c08e7 (127.23.194.1:43967), 5de46e59381f438b9436e35be05cb093 (127.23.194.3:37529)
I20260808 23:00:16.251744 3792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf4f0d6137d2413382a2530b21f66f82" candidate_uuid: "a0f3243b474d4e25866098bc36f944c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2ec16fd44742b788f43a95215c08e7" is_pre_election: true
I20260808 23:00:16.251744 4230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf4f0d6137d2413382a2530b21f66f82" candidate_uuid: "a0f3243b474d4e25866098bc36f944c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de46e59381f438b9436e35be05cb093" is_pre_election: true
I20260808 23:00:16.251863 3792 raft_consensus.cc:2454] T bf4f0d6137d2413382a2530b21f66f82 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0f3243b474d4e25866098bc36f944c6 in term 0.
I20260808 23:00:16.251863 4230 raft_consensus.cc:2454] T bf4f0d6137d2413382a2530b21f66f82 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0f3243b474d4e25866098bc36f944c6 in term 0.
I20260808 23:00:16.252008 3910 leader_election.cc:302] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5de46e59381f438b9436e35be05cb093, a0f3243b474d4e25866098bc36f944c6; no voters:
I20260808 23:00:16.252126 4371 raft_consensus.cc:2781] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:00:16.252182 4371 raft_consensus.cc:482] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:00:16.252206 4371 raft_consensus.cc:3037] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:16.252895 4371 raft_consensus.cc:504] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:16.253012 4371 leader_election.cc:288] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [CANDIDATE]: Term 1 election: Requested vote from peers 9c2ec16fd44742b788f43a95215c08e7 (127.23.194.1:43967), 5de46e59381f438b9436e35be05cb093 (127.23.194.3:37529)
I20260808 23:00:16.253163 3792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf4f0d6137d2413382a2530b21f66f82" candidate_uuid: "a0f3243b474d4e25866098bc36f944c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2ec16fd44742b788f43a95215c08e7"
I20260808 23:00:16.253185 4230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf4f0d6137d2413382a2530b21f66f82" candidate_uuid: "a0f3243b474d4e25866098bc36f944c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de46e59381f438b9436e35be05cb093"
I20260808 23:00:16.253248 3792 raft_consensus.cc:3037] T bf4f0d6137d2413382a2530b21f66f82 P 9c2ec16fd44742b788f43a95215c08e7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:16.253255 4230 raft_consensus.cc:3037] T bf4f0d6137d2413382a2530b21f66f82 P 5de46e59381f438b9436e35be05cb093 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:16.253857 3792 raft_consensus.cc:2454] T bf4f0d6137d2413382a2530b21f66f82 P 9c2ec16fd44742b788f43a95215c08e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0f3243b474d4e25866098bc36f944c6 in term 1.
I20260808 23:00:16.253964 4230 raft_consensus.cc:2454] T bf4f0d6137d2413382a2530b21f66f82 P 5de46e59381f438b9436e35be05cb093 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0f3243b474d4e25866098bc36f944c6 in term 1.
I20260808 23:00:16.254024 3911 leader_election.cc:302] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2ec16fd44742b788f43a95215c08e7, a0f3243b474d4e25866098bc36f944c6; no voters:
I20260808 23:00:16.254112 4371 raft_consensus.cc:2781] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:16.254215 4371 raft_consensus.cc:686] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [term 1 LEADER]: Becoming Leader. State: Replica: a0f3243b474d4e25866098bc36f944c6, State: Running, Role: LEADER
I20260808 23:00:16.254285 4371 consensus_queue.cc:237] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } }
I20260808 23:00:16.254853 3557 catalog_manager.cc:5495] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0f3243b474d4e25866098bc36f944c6 (127.23.194.2). New cstate: current_term: 1 leader_uuid: "a0f3243b474d4e25866098bc36f944c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0f3243b474d4e25866098bc36f944c6" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 } health_report { overall_health: UNKNOWN } } }
I20260808 23:00:16.311281 4353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:16.309260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46034 (local address 127.23.194.2:44439)
0808 23:00:16.309305 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:16.309307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:16.309316 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:00:16.309348 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:16.309349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:16.309359 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:16.309436 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:00:16.309537 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:16.310005 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:16.310686 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:16.310880 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:16.310933 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:16.311057 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:00:16.311090 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:00:16.311129 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:16.311181 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 23:00:16.311204 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 23:00:16.315335 4353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:16.313303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:52435 (local address 127.23.194.2:44439)
0808 23:00:16.313338 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:16.313340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:16.313409 (+ 69us) server_negotiation.cc:407] Connection header received
0808 23:00:16.313465 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:16.313466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:16.313475 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:16.313523 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:00:16.313631 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:16.314089 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:16.314766 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:16.314946 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:16.314998 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:16.315103 (+ 105us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:00:16.315130 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:00:16.315173 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:16.315221 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 23:00:16.315243 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":16}
I20260808 23:00:16.315343 4509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:16.313194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:44439 (local address 127.23.194.2:52435)
0808 23:00:16.313344 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:16.313353 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:16.313425 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:16.313525 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:16.313526 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:16.313527 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:00:16.313581 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:16.313582 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:16.314153 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:16.314154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:16.314699 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:16.314701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:16.315000 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:16.315001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:16.315050 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:16.315053 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:00:16.315182 (+ 129us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:00:16.315184 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 23:00:16.315221 (+ 37us) client_negotiation.cc:240] Negotiation successful
0808 23:00:16.315266 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":1,"thread_start_us":96,"threads_started":1}
I20260808 23:00:16.317318 4230 raft_consensus.cc:1261] T bf4f0d6137d2413382a2530b21f66f82 P 5de46e59381f438b9436e35be05cb093 [term 1 FOLLOWER]: Refusing update from remote peer a0f3243b474d4e25866098bc36f944c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:00:16.317351 3792 raft_consensus.cc:1261] T bf4f0d6137d2413382a2530b21f66f82 P 9c2ec16fd44742b788f43a95215c08e7 [term 1 FOLLOWER]: Refusing update from remote peer a0f3243b474d4e25866098bc36f944c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:00:16.317550 4511 consensus_queue.cc:1035] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5de46e59381f438b9436e35be05cb093" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 37529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:16.317679 4371 consensus_queue.cc:1035] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2ec16fd44742b788f43a95215c08e7" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 43967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:16.327754 3557 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:00:16.327844 3557 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:00:16.328380 3557 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb9422677c124ee5b7d5fc3f97515bd7: Sending DeleteTablet for 3 replicas of tablet bf4f0d6137d2413382a2530b21f66f82
I20260808 23:00:16.328593 3765 tablet_service.cc:1514] Processing DeleteTablet for tablet bf4f0d6137d2413382a2530b21f66f82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:16 UTC) from {username='oryx', principal='oryx/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:55212
I20260808 23:00:16.328629 4202 tablet_service.cc:1514] Processing DeleteTablet for tablet bf4f0d6137d2413382a2530b21f66f82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:16 UTC) from {username='oryx', principal='oryx/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:43354
I20260808 23:00:16.328732 4396 tablet_replica.cc:330] T bf4f0d6137d2413382a2530b21f66f82 P 5de46e59381f438b9436e35be05cb093: stopping tablet replica
I20260808 23:00:16.328722 3995 tablet_service.cc:1514] Processing DeleteTablet for tablet bf4f0d6137d2413382a2530b21f66f82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:16 UTC) from {username='oryx', principal='oryx/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:46010
I20260808 23:00:16.328734 4394 tablet_replica.cc:330] T bf4f0d6137d2413382a2530b21f66f82 P 9c2ec16fd44742b788f43a95215c08e7: stopping tablet replica
I20260808 23:00:16.328779 4396 raft_consensus.cc:2229] T bf4f0d6137d2413382a2530b21f66f82 P 5de46e59381f438b9436e35be05cb093 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:00:16.328783 4394 raft_consensus.cc:2229] T bf4f0d6137d2413382a2530b21f66f82 P 9c2ec16fd44742b788f43a95215c08e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:00:16.328814 4394 raft_consensus.cc:2258] T bf4f0d6137d2413382a2530b21f66f82 P 9c2ec16fd44742b788f43a95215c08e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:16.328814 4396 raft_consensus.cc:2258] T bf4f0d6137d2413382a2530b21f66f82 P 5de46e59381f438b9436e35be05cb093 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:16.328823 4397 tablet_replica.cc:330] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6: stopping tablet replica
I20260808 23:00:16.328881 4397 raft_consensus.cc:2229] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:00:16.328935 4397 raft_consensus.cc:2258] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:16.329111 4394 ts_tablet_manager.cc:1905] T bf4f0d6137d2413382a2530b21f66f82 P 9c2ec16fd44742b788f43a95215c08e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:16.329185 4396 ts_tablet_manager.cc:1905] T bf4f0d6137d2413382a2530b21f66f82 P 5de46e59381f438b9436e35be05cb093: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:16.329255 4397 ts_tablet_manager.cc:1905] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:16.330687 4397 ts_tablet_manager.cc:1918] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:00:16.330713 4396 ts_tablet_manager.cc:1918] T bf4f0d6137d2413382a2530b21f66f82 P 5de46e59381f438b9436e35be05cb093: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:00:16.330754 4397 log.cc:1196] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/bf4f0d6137d2413382a2530b21f66f82
I20260808 23:00:16.330765 4396 log.cc:1196] T bf4f0d6137d2413382a2530b21f66f82 P 5de46e59381f438b9436e35be05cb093: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/bf4f0d6137d2413382a2530b21f66f82
I20260808 23:00:16.330914 4397 ts_tablet_manager.cc:1939] T bf4f0d6137d2413382a2530b21f66f82 P a0f3243b474d4e25866098bc36f944c6: Deleting consensus metadata
I20260808 23:00:16.330902 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 3673
I20260808 23:00:16.330969 4396 ts_tablet_manager.cc:1939] T bf4f0d6137d2413382a2530b21f66f82 P 5de46e59381f438b9436e35be05cb093: Deleting consensus metadata
I20260808 23:00:16.331043 4394 ts_tablet_manager.cc:1918] T bf4f0d6137d2413382a2530b21f66f82 P 9c2ec16fd44742b788f43a95215c08e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:00:16.331080 4394 log.cc:1196] T bf4f0d6137d2413382a2530b21f66f82 P 9c2ec16fd44742b788f43a95215c08e7: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786229981209642-24328-0/minicluster-data/ts-0/wal/wals/bf4f0d6137d2413382a2530b21f66f82
I20260808 23:00:16.331264 4394 ts_tablet_manager.cc:1939] T bf4f0d6137d2413382a2530b21f66f82 P 9c2ec16fd44742b788f43a95215c08e7: Deleting consensus metadata
I20260808 23:00:16.334024 3525 catalog_manager.cc:4841] TS a0f3243b474d4e25866098bc36f944c6 (127.23.194.2:44439): tablet bf4f0d6137d2413382a2530b21f66f82 (table test-table [id=d43e57bb05a54facbdb6ce049758b20b]) successfully deleted
I20260808 23:00:16.336540 3526 catalog_manager.cc:4841] TS 5de46e59381f438b9436e35be05cb093 (127.23.194.3:37529): tablet bf4f0d6137d2413382a2530b21f66f82 (table test-table [id=d43e57bb05a54facbdb6ce049758b20b]) successfully deleted
W20260808 23:00:16.343813 3527 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.194.1:43967 (error 108)
I20260808 23:00:16.343999 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 3884
I20260808 23:00:16.350822 4519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:16.350588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:43967 (local address 127.0.0.1:55234)
0808 23:00:16.350731 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:16.350778 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.1:43967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":49,"threads_started":1}
W20260808 23:00:16.350930 3527 catalog_manager.cc:4584] TS 9c2ec16fd44742b788f43a95215c08e7 (127.23.194.1:43967): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bf4f0d6137d2413382a2530b21f66f82: Network error: Client connection negotiation failed: client connection to 127.23.194.1:43967: connect: Connection refused (error 111)
I20260808 23:00:16.358603 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 4107
I20260808 23:00:16.369104 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 3497
2026-08-08T23:00:16Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3022 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:00:16 dist-test-slave-f0tn krb5kdc[4522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:00:16 dist-test-slave-f0tn krb5kdc[4522](info): set up 2 sockets
Aug 08 23:00:16 dist-test-slave-f0tn krb5kdc[4522](info): commencing operation
krb5kdc: starting...
W20260808 23:00:17.518277 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:00:17 dist-test-slave-f0tn krb5kdc[4522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:00:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:00:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:00:17.636050 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:37593
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:38723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:37593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:17.716956 4539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:17.717178 4539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:17.717208 4539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:17.718357 4539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:00:17.718397 4539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:17.718415 4539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:17.718431 4539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:00:17.718446 4539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:00:17.719894 4539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:38723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:37593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:37593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:17.720148 4539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:17.720348 4539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:17.725116 4547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:17.725121 4546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:17.725111 4549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:17.725172 4539 server_base.cc:1034] running on GCE node
I20260808 23:00:17.725693 4539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:17.725947 4539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:17.727084 4539 hybrid_clock.cc:648] HybridClock initialized: now 1786230017727068 us; error 29 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T23:00:17Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1356 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:00:17 dist-test-slave-f0tn krb5kdc[4554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:00:17 dist-test-slave-f0tn krb5kdc[4554](info): set up 2 sockets
Aug 08 23:00:17 dist-test-slave-f0tn krb5kdc[4554](info): commencing operation
krb5kdc: starting...
W20260808 23:00:18.854912 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:00:18 dist-test-slave-f0tn krb5kdc[4554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:00:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:00:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:00:18.966087 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:43137
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:35765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:43137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:19.045076 4576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:19.045277 4576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:19.045307 4576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:19.046417 4576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:00:19.046458 4576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:19.046478 4576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:19.046492 4576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:00:19.046509 4576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:00:19.047926 4576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:35765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:43137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:43137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:19.048166 4576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:19.048359 4576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:19.052839 4586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:19.052845 4587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:19.052855 4590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:19.052965 4576 server_base.cc:1034] running on GCE node
I20260808 23:00:19.053293 4576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:19.053583 4576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:19.054737 4576 hybrid_clock.cc:648] HybridClock initialized: now 1786230019054726 us; error 39 us; skew 500 ppm
Aug 08 23:00:19 dist-test-slave-f0tn krb5kdc[4554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230019, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:19.057687 4576 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:00:19.058131 4576 webserver.cc:460] Webserver started at http://127.23.194.62:33883/ using document root <none> and password file <none>
I20260808 23:00:19.058337 4576 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:19.058403 4576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:19.058537 4576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:19.059610 4576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "bb7acf3d67f34bc29247faabefac2f98"
format_stamp: "Formatted at 2026-08-08 23:00:19 on dist-test-slave-f0tn"
server_key: "1910ac8555cb9fd16f85486fbb7af979"
server_key_iv: "e2a4f7e86d7d8890bf77c2860fbc3bad"
server_key_version: "encryptionkey@0"
I20260808 23:00:19.059900 4576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "bb7acf3d67f34bc29247faabefac2f98"
format_stamp: "Formatted at 2026-08-08 23:00:19 on dist-test-slave-f0tn"
server_key: "1910ac8555cb9fd16f85486fbb7af979"
server_key_iv: "e2a4f7e86d7d8890bf77c2860fbc3bad"
server_key_version: "encryptionkey@0"
I20260808 23:00:19.061029 4576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:00:19.061666 4601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:19.061815 4576 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:19.061898 4576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "bb7acf3d67f34bc29247faabefac2f98"
format_stamp: "Formatted at 2026-08-08 23:00:19 on dist-test-slave-f0tn"
server_key: "1910ac8555cb9fd16f85486fbb7af979"
server_key_iv: "e2a4f7e86d7d8890bf77c2860fbc3bad"
server_key_version: "encryptionkey@0"
I20260808 23:00:19.061951 4576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:19.069824 4576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:19.071784 4576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:19.071908 4576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:19.075577 4576 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:43137
I20260808 23:00:19.075623 4678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:43137 every 8 connection(s)
I20260808 23:00:19.075924 4576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:00:19.076758 4679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:19.079089 4679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98: Bootstrap starting.
I20260808 23:00:19.079638 4679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:19.079901 4679 log.cc:824] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:19.080175 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 4576
I20260808 23:00:19.080283 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:00:19.080493 24328 external_mini_cluster.cc:1428] Setting key 333a86af7fe1b5fb45af62459150d353
I20260808 23:00:19.080653 4679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98: No bootstrap required, opened a new log
I20260808 23:00:19.082863 4679 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb7acf3d67f34bc29247faabefac2f98" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 43137 } }
I20260808 23:00:19.083001 4679 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:19.083024 4679 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb7acf3d67f34bc29247faabefac2f98, State: Initialized, Role: FOLLOWER
I20260808 23:00:19.083133 4679 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb7acf3d67f34bc29247faabefac2f98" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 43137 } }
I20260808 23:00:19.083209 4679 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:00:19.083245 4679 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:00:19.083287 4679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:19.083789 4679 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb7acf3d67f34bc29247faabefac2f98" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 43137 } }
I20260808 23:00:19.083877 4679 leader_election.cc:302] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb7acf3d67f34bc29247faabefac2f98; no voters:
I20260808 23:00:19.084040 4679 leader_election.cc:288] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:00:19.084082 4684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:19.084203 4684 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [term 1 LEADER]: Becoming Leader. State: Replica: bb7acf3d67f34bc29247faabefac2f98, State: Running, Role: LEADER
I20260808 23:00:19.084285 4684 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb7acf3d67f34bc29247faabefac2f98" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 43137 } }
I20260808 23:00:19.084306 4679 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:00:19.085887 4687 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb7acf3d67f34bc29247faabefac2f98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb7acf3d67f34bc29247faabefac2f98" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 43137 } } }
I20260808 23:00:19.085971 4687 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:19.086046 4686 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb7acf3d67f34bc29247faabefac2f98. Latest consensus state: current_term: 1 leader_uuid: "bb7acf3d67f34bc29247faabefac2f98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb7acf3d67f34bc29247faabefac2f98" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 43137 } } }
I20260808 23:00:19.086127 4686 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [sys.catalog]: This master's current role is: LEADER
W20260808 23:00:19.086725 4712 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:00:19.087132 4712 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:00:19.087217 4706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:00:19.087419 4706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 23:00:19 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:19.089358 4706 catalog_manager.cc:1347] Generated new cluster ID: 83fa9539f43346bc8a94ec939fa8c7f5
I20260808 23:00:19.089437 4706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:00:19.091114 4683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:19.081002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47728 (local address 127.23.194.62:43137)
0808 23:00:19.081148 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:19.081157 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:19.081182 (+ 25us) server_negotiation.cc:407] Connection header received
0808 23:00:19.081957 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:19.081964 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:19.082012 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:19.082121 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:19.082868 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.084128 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.085006 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.085473 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.085572 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:19.088044 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:19.088048 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:19.088048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:19.088060 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:19.089755 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:19.090053 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:19.090056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:19.090056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:19.090148 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:19.090320 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:19.090321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:19.090321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:19.090537 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:19.090667 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:19.090788 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 23:00:19.090912 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
I20260808 23:00:19.101269 4706 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:00:19.101976 4706 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:00:19.114388 4706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98: Generated new TSK 0
I20260808 23:00:19.114550 4706 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
I20260808 23:00:19.139952 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:43137
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:35765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:19.220008 4720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:19.220216 4720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:19.220247 4720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:19.221405 4720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:19.221442 4720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:19.221477 4720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:00:19.222911 4720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:35765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:43137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:19.223150 4720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:19.223362 4720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:19.228156 4732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:19.228147 4729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:19.228248 4728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:19.228384 4720 server_base.cc:1034] running on GCE node
I20260808 23:00:19.228575 4720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:19.228766 4720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:19.229895 4720 hybrid_clock.cc:648] HybridClock initialized: now 1786230019229882 us; error 28 us; skew 500 ppm
Aug 08 23:00:19 dist-test-slave-f0tn krb5kdc[4554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230019, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:19.232381 4720 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 23:00:19.232796 4720 webserver.cc:460] Webserver started at http://127.23.194.1:44257/ using document root <none> and password file <none>
I20260808 23:00:19.233011 4720 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:19.233054 4720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:19.233178 4720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:19.234197 4720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "287ccc9da4cf4545bd2209bfbb247b06"
format_stamp: "Formatted at 2026-08-08 23:00:19 on dist-test-slave-f0tn"
server_key: "3c9a5db1f640081ad2b22881e91773d6"
server_key_iv: "d222dba99d47cadb48e4d3e515236504"
server_key_version: "encryptionkey@0"
I20260808 23:00:19.234499 4720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "287ccc9da4cf4545bd2209bfbb247b06"
format_stamp: "Formatted at 2026-08-08 23:00:19 on dist-test-slave-f0tn"
server_key: "3c9a5db1f640081ad2b22881e91773d6"
server_key_iv: "d222dba99d47cadb48e4d3e515236504"
server_key_version: "encryptionkey@0"
I20260808 23:00:19.235726 4720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:00:19.236373 4745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:19.236547 4720 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:19.236631 4720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "287ccc9da4cf4545bd2209bfbb247b06"
format_stamp: "Formatted at 2026-08-08 23:00:19 on dist-test-slave-f0tn"
server_key: "3c9a5db1f640081ad2b22881e91773d6"
server_key_iv: "d222dba99d47cadb48e4d3e515236504"
server_key_version: "encryptionkey@0"
I20260808 23:00:19.236686 4720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:19.251371 4720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:19.252964 4720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:19.253067 4720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:19.253242 4720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:19.253533 4720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:19.253567 4720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:19.253587 4720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:19.253604 4720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:19.258867 4720 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:44839
I20260808 23:00:19.258929 4929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:44839 every 8 connection(s)
I20260808 23:00:19.259202 4720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
Aug 08 23:00:19 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230019, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:19.265174 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 4720
I20260808 23:00:19.265265 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:00:19.265503 24328 external_mini_cluster.cc:1428] Setting key 16b0779bdc6a2230f89802abc33d59fc
I20260808 23:00:19.268050 4683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:19.259840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:36915 (local address 127.23.194.62:43137)
0808 23:00:19.259895 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:19.259898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:19.260073 (+ 175us) server_negotiation.cc:407] Connection header received
0808 23:00:19.260567 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:19.260569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:19.260582 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:19.260651 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:19.261586 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.262127 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.263074 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.263284 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.263339 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:19.265371 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:19.265373 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:19.265394 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:19.265403 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:19.266846 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:19.267197 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:19.267197 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:19.267198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:19.267258 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:19.267452 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:19.267453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:19.267453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:19.267578 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:19.267635 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:19.267856 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 23:00:19.267945 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 23:00:19.268173 4933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:19.259814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.1:36915)
0808 23:00:19.259962 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:19.259997 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:19.260465 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:19.260701 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:19.260705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:19.261248 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:19.261497 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:19.261503 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.262212 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.262214 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:19.262992 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:19.262994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.263353 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.263354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:19.263469 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:19.263931 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:19.263939 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:19.265292 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:19.266920 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:19.266926 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:19.266928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:19.267132 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:19.267311 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:19.267311 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:19.267311 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:19.267397 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:19.267633 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:19.267641 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:19.267751 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 23:00:19.267825 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 23:00:19.267946 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
I20260808 23:00:19.268618 4930 heartbeater.cc:347] Connected to a master server at 127.23.194.62:43137
I20260808 23:00:19.268712 4930 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:19.268885 4930 heartbeater.cc:511] Master 127.23.194.62:43137 requested a full tablet report, sending...
I20260808 23:00:19.269464 4629 ts_manager.cc:194] Registered new tserver with Master: 287ccc9da4cf4545bd2209bfbb247b06 (127.23.194.1:44839)
I20260808 23:00:19.270273 4629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:36915
WARNING: no policy specified for kudu/127.23.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
I20260808 23:00:19.289371 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:43137
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:35765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:19.368829 4940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:19.369028 4940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:19.369056 4940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:19.370209 4940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:19.370251 4940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:19.370285 4940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:00:19.371672 4940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:35765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:43137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:19.371912 4940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:19.372112 4940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:19.376866 4953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:19.376868 4949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:19.376868 4948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:19.377101 4940 server_base.cc:1034] running on GCE node
I20260808 23:00:19.377310 4940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:19.377544 4940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:19.378681 4940 hybrid_clock.cc:648] HybridClock initialized: now 1786230019378667 us; error 30 us; skew 500 ppm
Aug 08 23:00:19 dist-test-slave-f0tn krb5kdc[4554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230019, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:19.381026 4940 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 23:00:19.381439 4940 webserver.cc:460] Webserver started at http://127.23.194.2:39205/ using document root <none> and password file <none>
I20260808 23:00:19.381618 4940 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:19.381659 4940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:19.381764 4940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:19.382704 4940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "1a5a5871f690453d9deb07bfe2bb14ca"
format_stamp: "Formatted at 2026-08-08 23:00:19 on dist-test-slave-f0tn"
server_key: "3d4636abade7f7844a142949f0806b6c"
server_key_iv: "54548c998e1e315f09280beb57b8a27b"
server_key_version: "encryptionkey@0"
I20260808 23:00:19.382984 4940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "1a5a5871f690453d9deb07bfe2bb14ca"
format_stamp: "Formatted at 2026-08-08 23:00:19 on dist-test-slave-f0tn"
server_key: "3d4636abade7f7844a142949f0806b6c"
server_key_iv: "54548c998e1e315f09280beb57b8a27b"
server_key_version: "encryptionkey@0"
I20260808 23:00:19.384191 4940 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:00:19.384889 4968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:19.385032 4940 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:19.385113 4940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "1a5a5871f690453d9deb07bfe2bb14ca"
format_stamp: "Formatted at 2026-08-08 23:00:19 on dist-test-slave-f0tn"
server_key: "3d4636abade7f7844a142949f0806b6c"
server_key_iv: "54548c998e1e315f09280beb57b8a27b"
server_key_version: "encryptionkey@0"
I20260808 23:00:19.385188 4940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:19.392333 4940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:19.393957 4940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:19.394076 4940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:19.394268 4940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:19.394529 4940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:19.394560 4940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:19.394580 4940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:19.394603 4940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:19.399951 4940 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:34227
I20260808 23:00:19.399995 5142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:34227 every 8 connection(s)
I20260808 23:00:19.400259 4940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:00:19.403792 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 4940
I20260808 23:00:19.403894 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:00:19.404100 24328 external_mini_cluster.cc:1428] Setting key 176c1c8187cdddae603e0363daaa4146
Aug 08 23:00:19 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230019, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:19.409976 4683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:19.400893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:41147 (local address 127.23.194.62:43137)
0808 23:00:19.400950 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:19.400952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:19.401124 (+ 172us) server_negotiation.cc:407] Connection header received
0808 23:00:19.401615 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:19.401616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:19.401626 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:19.401686 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:19.402532 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.403024 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.404111 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.405369 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.405447 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:19.407558 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:19.407559 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:19.407560 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:19.407567 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:19.408705 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:19.409052 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:19.409053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:19.409053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:19.409110 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:19.409329 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:19.409329 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:19.409330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:19.409486 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:19.409545 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:19.409807 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 23:00:19.409887 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 23:00:19.410123 5147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:19.400852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.2:41147)
0808 23:00:19.401009 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:19.401048 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:19.401540 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:19.401752 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:19.401755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:19.402286 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:19.402462 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:19.402466 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.403083 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.403083 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:19.404040 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:19.404043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.405464 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.405465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:19.405604 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:19.406056 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:19.406066 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:19.407473 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:19.408777 (+ 1304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:19.408784 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:19.408786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:19.408988 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:19.409162 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:19.409163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:19.409164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:19.409273 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:19.409542 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:19.409549 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:19.409686 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 23:00:19.409772 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 23:00:19.409919 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
I20260808 23:00:19.410617 5145 heartbeater.cc:347] Connected to a master server at 127.23.194.62:43137
I20260808 23:00:19.410724 5145 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:19.410907 5145 heartbeater.cc:511] Master 127.23.194.62:43137 requested a full tablet report, sending...
I20260808 23:00:19.411250 4629 ts_manager.cc:194] Registered new tserver with Master: 1a5a5871f690453d9deb07bfe2bb14ca (127.23.194.2:34227)
I20260808 23:00:19.411605 4629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:41147
WARNING: no policy specified for kudu/127.23.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
I20260808 23:00:19.427719 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:43137
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:35765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:19.507596 5150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:19.507791 5150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:19.507812 5150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:19.508915 5150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:19.508956 5150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:19.508985 5150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:00:19.510470 5150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:35765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:43137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:19.510699 5150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:19.510896 5150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:19.515553 5156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:19.515568 5157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:19.515580 5159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:19.515738 5150 server_base.cc:1034] running on GCE node
I20260808 23:00:19.515977 5150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:19.516202 5150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:19.517339 5150 hybrid_clock.cc:648] HybridClock initialized: now 1786230019517325 us; error 32 us; skew 500 ppm
Aug 08 23:00:19 dist-test-slave-f0tn krb5kdc[4554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230019, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:19.519727 5150 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 23:00:19.520138 5150 webserver.cc:460] Webserver started at http://127.23.194.3:35061/ using document root <none> and password file <none>
I20260808 23:00:19.520345 5150 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:19.520392 5150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:19.520519 5150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:19.521541 5150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "b9d8d63f31e64cbc974a720921d5a770"
format_stamp: "Formatted at 2026-08-08 23:00:19 on dist-test-slave-f0tn"
server_key: "80813a41d34310b3136b5b397344ff8a"
server_key_iv: "64e218c7bcf4298d51059df7f97eb635"
server_key_version: "encryptionkey@0"
I20260808 23:00:19.521876 5150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "b9d8d63f31e64cbc974a720921d5a770"
format_stamp: "Formatted at 2026-08-08 23:00:19 on dist-test-slave-f0tn"
server_key: "80813a41d34310b3136b5b397344ff8a"
server_key_iv: "64e218c7bcf4298d51059df7f97eb635"
server_key_version: "encryptionkey@0"
I20260808 23:00:19.523092 5150 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:00:19.523720 5168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:19.523870 5150 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:19.523949 5150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "b9d8d63f31e64cbc974a720921d5a770"
format_stamp: "Formatted at 2026-08-08 23:00:19 on dist-test-slave-f0tn"
server_key: "80813a41d34310b3136b5b397344ff8a"
server_key_iv: "64e218c7bcf4298d51059df7f97eb635"
server_key_version: "encryptionkey@0"
I20260808 23:00:19.524008 5150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:19.530880 5150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:19.532791 5150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:19.532912 5150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:19.533085 5150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:19.533353 5150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:19.533413 5150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:19.533443 5150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:19.533468 5150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:19.538861 5150 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:46819
I20260808 23:00:19.538897 5353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:46819 every 8 connection(s)
I20260808 23:00:19.539161 5150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 23:00:19.542503 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 5150
I20260808 23:00:19.542598 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:00:19.542765 24328 external_mini_cluster.cc:1428] Setting key aaab106bf9693a9939417113596ed5a0
Aug 08 23:00:19 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230019, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:19.547223 4683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:19.539814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:44063 (local address 127.23.194.62:43137)
0808 23:00:19.539870 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:19.539872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:19.540023 (+ 151us) server_negotiation.cc:407] Connection header received
0808 23:00:19.540478 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:19.540479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:19.540488 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:19.540540 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:19.541356 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.541871 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.542867 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.543150 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.543207 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:19.545128 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:19.545129 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:19.545129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:19.545146 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:19.546223 (+ 1077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:19.546550 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:19.546551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:19.546551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:19.546597 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:19.546773 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:19.546773 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:19.546773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:19.546864 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:19.546908 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:19.547095 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 23:00:19.547160 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 23:00:19.547354 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:19.539771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.3:44063)
0808 23:00:19.539910 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:19.539948 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:19.540400 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:19.540588 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:19.540591 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:19.541117 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:19.541291 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:19.541296 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.541935 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.541936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:19.542790 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:19.542794 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.543215 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.543217 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:19.543334 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:19.543775 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:19.543783 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:19.545054 (+ 1271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:19.546285 (+ 1231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:19.546291 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:19.546292 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:19.546494 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:19.546639 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:19.546640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:19.546640 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:19.546732 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:19.546904 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:19.546907 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:19.547006 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 23:00:19.547080 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 23:00:19.547190 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
I20260808 23:00:19.547753 5354 heartbeater.cc:347] Connected to a master server at 127.23.194.62:43137
I20260808 23:00:19.547832 5354 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:19.547994 5354 heartbeater.cc:511] Master 127.23.194.62:43137 requested a full tablet report, sending...
I20260808 23:00:19.548310 4629 ts_manager.cc:194] Registered new tserver with Master: b9d8d63f31e64cbc974a720921d5a770 (127.23.194.3:46819)
I20260808 23:00:19.548660 4629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:44063
I20260808 23:00:19.554332 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.1.keytab.
Entry for principal oryx/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.1.keytab.
WARNING: no policy specified for oryx/127.23.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.2.keytab.
Entry for principal oryx/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.2.keytab.
WARNING: no policy specified for oryx/127.23.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.3.keytab.
Entry for principal oryx/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.3.keytab.
I20260808 23:00:19.615309 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 4720
I20260808 23:00:19.627852 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 4940
I20260808 23:00:19.641948 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 5150
I20260808 23:00:19.649478 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 4576
I20260808 23:00:19.659786 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:43137
--webserver_interface=127.23.194.62
--webserver_port=33883
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:35765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:43137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:19.739416 5364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:19.739634 5364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:19.739665 5364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:19.740788 5364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:00:19.740828 5364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:19.740847 5364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:19.740863 5364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:00:19.740880 5364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:00:19.742437 5364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:35765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:43137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:43137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=33883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:19.742689 5364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:19.742897 5364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:19.747418 5372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:19.747440 5370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:19.747499 5364 server_base.cc:1034] running on GCE node
W20260808 23:00:19.747442 5374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:19.747828 5364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:19.748090 5364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:19.749228 5364 hybrid_clock.cc:648] HybridClock initialized: now 1786230019749216 us; error 27 us; skew 500 ppm
Aug 08 23:00:19 dist-test-slave-f0tn krb5kdc[4554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230019, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:19.751487 5364 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:00:19.751863 5364 webserver.cc:460] Webserver started at http://127.23.194.62:33883/ using document root <none> and password file <none>
I20260808 23:00:19.752059 5364 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:19.752122 5364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:19.753340 5364 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 23:00:19.753957 5383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:19.754098 5364 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:19.754189 5364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "bb7acf3d67f34bc29247faabefac2f98"
format_stamp: "Formatted at 2026-08-08 23:00:19 on dist-test-slave-f0tn"
server_key: "1910ac8555cb9fd16f85486fbb7af979"
server_key_iv: "e2a4f7e86d7d8890bf77c2860fbc3bad"
server_key_version: "encryptionkey@0"
I20260808 23:00:19.754482 5364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:19.772326 5364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:19.773976 5364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:19.774099 5364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:19.777784 5364 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:43137
I20260808 23:00:19.777817 5463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:43137 every 8 connection(s)
I20260808 23:00:19.778154 5364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:00:19.779512 5465 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:00:19.780136 5465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98: Bootstrap starting.
I20260808 23:00:19.784212 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 5364
I20260808 23:00:19.784473 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:44839
--local_ip_for_outbound_sockets=127.23.194.1
--tserver_master_addrs=127.23.194.62:43137
--webserver_port=44257
--webserver_interface=127.23.194.1
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:35765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.194.1
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:00:19.788796 5465 log.cc:824] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:19.790328 5465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:00:19.790571 5465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98: Bootstrap complete.
I20260808 23:00:19.792646 5465 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb7acf3d67f34bc29247faabefac2f98" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 43137 } }
I20260808 23:00:19.792872 5465 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb7acf3d67f34bc29247faabefac2f98, State: Initialized, Role: FOLLOWER
I20260808 23:00:19.793007 5465 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb7acf3d67f34bc29247faabefac2f98" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 43137 } }
I20260808 23:00:19.793071 5465 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:00:19.793112 5465 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:00:19.793156 5465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [term 1 FOLLOWER]: Advancing to term 2
I20260808 23:00:19.794131 5465 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb7acf3d67f34bc29247faabefac2f98" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 43137 } }
I20260808 23:00:19.794217 5465 leader_election.cc:302] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb7acf3d67f34bc29247faabefac2f98; no voters:
I20260808 23:00:19.794366 5465 leader_election.cc:288] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 23:00:19.794413 5470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 23:00:19.794534 5470 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [term 2 LEADER]: Becoming Leader. State: Replica: bb7acf3d67f34bc29247faabefac2f98, State: Running, Role: LEADER
I20260808 23:00:19.794618 5470 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb7acf3d67f34bc29247faabefac2f98" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 43137 } }
I20260808 23:00:19.794625 5465 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:00:19.794924 5473 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bb7acf3d67f34bc29247faabefac2f98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb7acf3d67f34bc29247faabefac2f98" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 43137 } } }
I20260808 23:00:19.794929 5470 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb7acf3d67f34bc29247faabefac2f98. Latest consensus state: current_term: 2 leader_uuid: "bb7acf3d67f34bc29247faabefac2f98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb7acf3d67f34bc29247faabefac2f98" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 43137 } } }
I20260808 23:00:19.795028 5473 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:19.795035 5470 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:19.796324 5499 catalog_manager.cc:1259] Loaded cluster ID: 83fa9539f43346bc8a94ec939fa8c7f5
I20260808 23:00:19.796370 5499 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98: loading cluster ID for follower catalog manager: success
I20260808 23:00:19.797504 5499 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98: acquiring CA information for follower catalog manager: success
I20260808 23:00:19.797828 5499 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 23:00:19.797902 5484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:00:19.798031 5484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:00:19.798108 5484 catalog_manager.cc:1259] Loaded cluster ID: 83fa9539f43346bc8a94ec939fa8c7f5
I20260808 23:00:19.798139 5484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:00:19.798326 5484 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:00:19.798409 5484 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bb7acf3d67f34bc29247faabefac2f98: Loaded TSK: 0
I20260808 23:00:19.798564 5484 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 23:00:19.868336 5468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:19.868551 5468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:19.868583 5468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:19.869783 5468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:19.869828 5468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:19.869879 5468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:00:19.871352 5468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:35765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.1.keytab
--principal=oryx/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:44839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=44257
--tserver_master_addrs=127.23.194.62:43137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:19.871603 5468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:19.871824 5468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:19.876761 5511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:19.876773 5508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:19.876845 5517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:19.877115 5468 server_base.cc:1034] running on GCE node
I20260808 23:00:19.877319 5468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:19.877583 5468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:19.878733 5468 hybrid_clock.cc:648] HybridClock initialized: now 1786230019878717 us; error 32 us; skew 500 ppm
Aug 08 23:00:19 dist-test-slave-f0tn krb5kdc[4554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230019, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:19.881419 5468 init.cc:377] Logged in from keytab as oryx/127.23.194.1@KRBTEST.COM (short username oryx)
I20260808 23:00:19.881839 5468 webserver.cc:460] Webserver started at http://127.23.194.1:44257/ using document root <none> and password file <none>
I20260808 23:00:19.882055 5468 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:19.882112 5468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:19.883440 5468 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:00:19.884030 5527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:19.884176 5468 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:19.884258 5468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "287ccc9da4cf4545bd2209bfbb247b06"
format_stamp: "Formatted at 2026-08-08 23:00:19 on dist-test-slave-f0tn"
server_key: "3c9a5db1f640081ad2b22881e91773d6"
server_key_iv: "d222dba99d47cadb48e4d3e515236504"
server_key_version: "encryptionkey@0"
I20260808 23:00:19.884555 5468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:19.892112 5468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:19.894083 5468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:19.894205 5468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:19.894394 5468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:19.894680 5468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:19.894712 5468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:19.894739 5468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:19.894762 5468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:19.900331 5468 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:44839
I20260808 23:00:19.900381 5686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:44839 every 8 connection(s)
I20260808 23:00:19.900689 5468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
Aug 08 23:00:19 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:19 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:19.908588 5695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:19.901409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.1:42191)
0808 23:00:19.901654 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:19.901695 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:19.902174 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:19.902553 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:19.902557 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:19.903086 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:19.903274 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:19.903278 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.904359 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.904360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:19.905117 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:19.905120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.905741 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.905743 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:19.905862 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:19.906310 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:19.906317 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:19.908392 (+ 2075us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":159,"threads_started":1}
W20260808 23:00:19.908736 5693 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:19.901421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:42191 (local address 127.23.194.62:43137)
0808 23:00:19.901591 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:19.901600 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:19.901801 (+ 201us) server_negotiation.cc:407] Connection header received
0808 23:00:19.902355 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:19.902360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:19.902400 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:19.902509 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:19.903391 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.904266 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.905209 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.905663 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.905756 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:19.908549 (+ 2793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.1:42191: BlockingRecv error: recv got EOF from 127.23.194.1:42191 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
W20260808 23:00:19.909165 5687 heartbeater.cc:644] Failed to heartbeat to 127.23.194.62:43137 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.194.62:43137: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
I20260808 23:00:19.910070 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 5468
I20260808 23:00:19.910347 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:34227
--local_ip_for_outbound_sockets=127.23.194.2
--tserver_master_addrs=127.23.194.62:43137
--webserver_port=39205
--webserver_interface=127.23.194.2
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:35765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.194.2
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:00:19 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:19 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:19.913668 5695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:19.909407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.1:33835)
0808 23:00:19.909438 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:19.909448 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:19.909490 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:19.909603 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:19.909603 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:19.909707 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:19.909745 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:19.909746 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.910643 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.910645 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:19.911281 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:19.911284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.911747 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.911748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:19.911811 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:19.912184 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:19.912189 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:19.913609 (+ 1420us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":15}
W20260808 23:00:19.913774 5693 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:19.909441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:33835 (local address 127.23.194.62:43137)
0808 23:00:19.909486 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:19.909488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:19.909506 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:00:19.909529 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:19.909530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:19.909542 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:19.909602 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:19.909801 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.910564 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.911368 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.911675 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.911740 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:19.913720 (+ 1980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.1:33835: BlockingRecv error: recv got EOF from 127.23.194.1:33835 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 08 23:00:19 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:19 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:19.917838 5695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:19.914085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.1:59795)
0808 23:00:19.914181 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:19.914191 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:19.914234 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:19.914343 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:19.914344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:19.914452 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:19.914475 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:19.914476 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.915130 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.915131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:19.915640 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:19.915641 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.915961 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.915963 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:19.916011 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:19.916363 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:19.916367 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:19.917784 (+ 1417us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":82}
W20260808 23:00:19.917903 5693 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:19.914144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:59795 (local address 127.23.194.62:43137)
0808 23:00:19.914188 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:19.914190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:19.914221 (+ 31us) server_negotiation.cc:407] Connection header received
0808 23:00:19.914285 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:19.914286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:19.914295 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:19.914330 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:19.914531 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.914963 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.915715 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:19.915906 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:19.915957 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:19.917858 (+ 1901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.1:59795: BlockingRecv error: recv got EOF from 127.23.194.1:59795 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260808 23:00:19.918139 5687 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 23:00:19.993574 5696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:19.993765 5696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:19.993786 5696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:19.994972 5696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:19.995040 5696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:19.995076 5696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:00:19.996696 5696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:35765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.2.keytab
--principal=oryx/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:34227
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=39205
--tserver_master_addrs=127.23.194.62:43137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:19.996918 5696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:19.997131 5696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:20.002007 5705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:20.001986 5703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:20.001963 5702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:20.002166 5696 server_base.cc:1034] running on GCE node
I20260808 23:00:20.002396 5696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:20.002626 5696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:20.003763 5696 hybrid_clock.cc:648] HybridClock initialized: now 1786230020003753 us; error 29 us; skew 500 ppm
Aug 08 23:00:20 dist-test-slave-f0tn krb5kdc[4554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230020, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:20.006309 5696 init.cc:377] Logged in from keytab as oryx/127.23.194.2@KRBTEST.COM (short username oryx)
I20260808 23:00:20.006745 5696 webserver.cc:460] Webserver started at http://127.23.194.2:39205/ using document root <none> and password file <none>
I20260808 23:00:20.006949 5696 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:20.007005 5696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:20.008251 5696 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:00:20.008780 5715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:20.008920 5696 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:20.009001 5696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "1a5a5871f690453d9deb07bfe2bb14ca"
format_stamp: "Formatted at 2026-08-08 23:00:19 on dist-test-slave-f0tn"
server_key: "3d4636abade7f7844a142949f0806b6c"
server_key_iv: "54548c998e1e315f09280beb57b8a27b"
server_key_version: "encryptionkey@0"
I20260808 23:00:20.009336 5696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:20.029549 5696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:20.031323 5696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:20.031451 5696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:20.031683 5696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:20.031985 5696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:20.032018 5696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:20.032056 5696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:20.032080 5696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:20.037799 5696 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:34227
I20260808 23:00:20.037850 5892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:34227 every 8 connection(s)
I20260808 23:00:20.038192 5696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
Aug 08 23:00:20 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:20 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 23:00:20.045516 5693 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:20.038913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:60501 (local address 127.23.194.62:43137)
0808 23:00:20.038975 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:20.038977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:20.039184 (+ 207us) server_negotiation.cc:407] Connection header received
0808 23:00:20.039692 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:20.039694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:20.039710 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:20.039762 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:20.040849 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.041343 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.042355 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.042559 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.042613 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:20.045472 (+ 2859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.2:60501: BlockingRecv error: recv got EOF from 127.23.194.2:60501 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 23:00:20.045552 5896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:20.038896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.2:60501)
0808 23:00:20.039068 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:20.039104 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:20.039596 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:20.039829 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:20.039832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:20.040543 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:20.040734 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:20.040740 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.041451 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.041452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:20.042270 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:20.042273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.042634 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.042635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:20.042751 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:20.043333 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:20.043341 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:20.045364 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
W20260808 23:00:20.045756 5894 heartbeater.cc:644] Failed to heartbeat to 127.23.194.62:43137 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.194.62:43137: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
I20260808 23:00:20.046682 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 5696
I20260808 23:00:20.046984 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:46819
--local_ip_for_outbound_sockets=127.23.194.3
--tserver_master_addrs=127.23.194.62:43137
--webserver_port=35061
--webserver_interface=127.23.194.3
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:35765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.194.3
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:00:20 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:20 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:20.049976 5896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:20.045951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.2:44101)
0808 23:00:20.045991 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:20.046001 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:20.046043 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:20.046136 (+ 93us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:20.046137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:20.046241 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:20.046281 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:20.046282 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.046971 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.046973 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:20.047516 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:20.047518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.047887 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.047888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:20.047931 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:20.048293 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:20.048297 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:20.049922 (+ 1625us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 23:00:20.050074 5693 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:20.046004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:44101 (local address 127.23.194.62:43137)
0808 23:00:20.046042 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:20.046043 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:20.046052 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:00:20.046085 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:20.046085 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:20.046092 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:20.046127 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:20.046327 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.046905 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.047589 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.047821 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.047883 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:20.050016 (+ 2133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.2:44101: BlockingRecv error: recv got EOF from 127.23.194.2:44101 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
Aug 08 23:00:20 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:20 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:20.054229 5896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:20.050201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.2:44463)
0808 23:00:20.050283 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:20.050293 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:20.050343 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:20.050449 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:20.050450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:20.050552 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:20.050577 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:20.050578 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.051333 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.051333 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:20.051792 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:20.051793 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.052198 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.052198 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:20.052230 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:20.052592 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:20.052595 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:20.054187 (+ 1592us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":65}
W20260808 23:00:20.054343 5693 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:20.050244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:44463 (local address 127.23.194.62:43137)
0808 23:00:20.050293 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:20.050295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:20.050331 (+ 36us) server_negotiation.cc:407] Connection header received
0808 23:00:20.050381 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:20.050381 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:20.050390 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:20.050442 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:20.050630 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.051265 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.051858 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.052132 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.052196 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:20.054286 (+ 2090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.2:44463: BlockingRecv error: recv got EOF from 127.23.194.2:44463 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260808 23:00:20.054493 5894 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 23:00:20.129505 5897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:20.129683 5897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:20.129704 5897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:20.130815 5897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:20.130852 5897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:20.130878 5897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:00:20.132282 5897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:35765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/krb5kdc/oryx_127.23.194.3.keytab
--principal=oryx/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:46819
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=35061
--tserver_master_addrs=127.23.194.62:43137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:20.132501 5897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:20.132700 5897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:20.137458 5906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:20.137603 5897 server_base.cc:1034] running on GCE node
W20260808 23:00:20.137457 5908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:20.137759 5905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:20.137962 5897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:20.138187 5897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:20.139317 5897 hybrid_clock.cc:648] HybridClock initialized: now 1786230020139306 us; error 30 us; skew 500 ppm
Aug 08 23:00:20 dist-test-slave-f0tn krb5kdc[4554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230020, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:20.141695 5897 init.cc:377] Logged in from keytab as oryx/127.23.194.3@KRBTEST.COM (short username oryx)
I20260808 23:00:20.142123 5897 webserver.cc:460] Webserver started at http://127.23.194.3:35061/ using document root <none> and password file <none>
I20260808 23:00:20.142338 5897 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:20.142407 5897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:20.143689 5897 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:00:20.144192 5918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:20.144346 5897 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:20.144420 5897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "b9d8d63f31e64cbc974a720921d5a770"
format_stamp: "Formatted at 2026-08-08 23:00:19 on dist-test-slave-f0tn"
server_key: "80813a41d34310b3136b5b397344ff8a"
server_key_iv: "64e218c7bcf4298d51059df7f97eb635"
server_key_version: "encryptionkey@0"
I20260808 23:00:20.144733 5897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:20.156599 5897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:20.158394 5897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:20.158522 5897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:20.158723 5897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:20.159011 5897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:20.159044 5897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:20.159066 5897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:20.159089 5897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:20.164517 5897 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:46819
I20260808 23:00:20.164575 6095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:46819 every 8 connection(s)
I20260808 23:00:20.164896 5897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
Aug 08 23:00:20 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:20 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 23:00:20.171787 5693 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:20.165718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:37915 (local address 127.23.194.62:43137)
0808 23:00:20.165779 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:20.165781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:20.165950 (+ 169us) server_negotiation.cc:407] Connection header received
0808 23:00:20.166438 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:20.166439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:20.166452 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:20.166505 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:20.167436 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.167921 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.168873 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.169060 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.169115 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:20.171730 (+ 2615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.3:37915: BlockingRecv error: recv got EOF from 127.23.194.3:37915 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 23:00:20.171828 6098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:20.165670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.3:37915)
0808 23:00:20.165823 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:20.165867 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:20.166337 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:20.166574 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:20.166578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:20.167152 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:20.167333 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:20.167339 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.168000 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.168001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:20.168786 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:20.168788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.169126 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.169127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:20.169249 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:20.169710 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:20.169718 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:20.171626 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
W20260808 23:00:20.172046 6096 heartbeater.cc:644] Failed to heartbeat to 127.23.194.62:43137 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.194.62:43137: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
I20260808 23:00:20.172963 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 5897
Aug 08 23:00:20 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:20 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:20.176539 6098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:20.172251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.3:53339)
0808 23:00:20.172293 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:20.172304 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:20.172348 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:20.172551 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:20.172552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:20.172678 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:20.172719 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:20.172720 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.173421 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.173423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:20.174084 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:20.174087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.174395 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.174396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:20.174461 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:20.174824 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:20.174828 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:20.176476 (+ 1648us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 23:00:20.176592 5693 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:20.172419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:53339 (local address 127.23.194.62:43137)
0808 23:00:20.172466 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:20.172467 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:20.172473 (+ 6us) server_negotiation.cc:407] Connection header received
0808 23:00:20.172489 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:20.172490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:20.172497 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:20.172541 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:20.172782 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.173327 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.174162 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.174335 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.174385 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:20.176541 (+ 2156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.3:53339: BlockingRecv error: recv got EOF from 127.23.194.3:53339 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 08 23:00:20 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:20.180258 6102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:20.173313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47744 (local address 127.23.194.62:43137)
0808 23:00:20.173606 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:20.173608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:20.173612 (+ 4us) server_negotiation.cc:407] Connection header received
0808 23:00:20.173626 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:20.173627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:20.173635 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:20.173679 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:20.174375 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.174836 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.175760 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.176011 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.176078 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:20.177280 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:20.177284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:20.177285 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:20.177295 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:20.178875 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:20.179233 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:20.179235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:20.179236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:20.179328 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:20.179513 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:20.179514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:20.179514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:20.179831 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:20.179946 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:20.180066 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 23:00:20.180166 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":88,"threads_started":1}
Aug 08 23:00:20 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:20.180819 6098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:20.176784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.3:36827)
0808 23:00:20.176868 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:20.176877 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:20.176915 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:20.177074 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:20.177075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:20.177198 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:20.177221 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:20.177223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.177828 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.177829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:20.178302 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:20.178304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.178608 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.178609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:20.178657 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:20.179020 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:20.179024 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:20.180760 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":70}
W20260808 23:00:20.180889 5693 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:20.176832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:36827 (local address 127.23.194.62:43137)
0808 23:00:20.176992 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:20.176993 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:20.177000 (+ 7us) server_negotiation.cc:407] Connection header received
0808 23:00:20.177016 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:20.177016 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:20.177022 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:20.177064 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:20.177280 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.177764 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.178366 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.178555 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.178605 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:20.180838 (+ 2233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.3:36827: BlockingRecv error: recv got EOF from 127.23.194.3:36827 (error 108)
Metrics: {"server-negotiator.queue_time_us":143}
W20260808 23:00:20.180922 6096 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 23:00:20 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:20 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:20.925863 6104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:20.918464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.1:57571)
0808 23:00:20.918617 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:20.918629 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:20.918683 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:20.918810 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:20.918811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:20.918976 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:20.919026 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:20.919028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.919621 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.919621 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:20.920150 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:20.920151 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.920451 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.920452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:20.920488 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:20.924052 (+ 3564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:20.924058 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:20.925796 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":67,"threads_started":1}
W20260808 23:00:20.925964 6105 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:20.918517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:57571 (local address 127.23.194.62:43137)
0808 23:00:20.918658 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:20.918660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:20.918669 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:00:20.918739 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:20.918740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:20.918749 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:20.918796 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:20.919075 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.919566 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.920214 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:20.920404 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:20.920446 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:20.925894 (+ 5448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.1:57571: BlockingRecv error: recv got EOF from 127.23.194.1:57571 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
Aug 08 23:00:21 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:21 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:21.059428 6107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:21.054887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.2:38539)
0808 23:00:21.055018 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:21.055029 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:21.055097 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:21.055260 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:21.055261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:21.055431 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:21.055477 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:21.055479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:21.056119 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:21.056121 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:21.056743 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:21.056746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:21.057091 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:21.057092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:21.057150 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:21.057666 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:21.057670 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:21.059361 (+ 1691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
W20260808 23:00:21.059504 6105 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:21.054981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:38539 (local address 127.23.194.62:43137)
0808 23:00:21.055048 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:21.055051 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:21.055080 (+ 29us) server_negotiation.cc:407] Connection header received
0808 23:00:21.055149 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:21.055149 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:21.055199 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:21.055248 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:21.055535 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:21.056033 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:21.056806 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:21.057016 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:21.057073 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:21.059450 (+ 2377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.2:38539: BlockingRecv error: recv got EOF from 127.23.194.2:38539 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 08 23:00:21 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:21 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:21.185745 6108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:21.181305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.3:35201)
0808 23:00:21.181437 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:21.181448 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:21.181506 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:21.181658 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:21.181659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:21.181849 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:21.181896 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:21.181898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:21.182564 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:21.182565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:21.183086 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:21.183088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:21.183437 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:21.183438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:21.183493 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:21.183913 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:21.183917 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:21.185677 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":57,"threads_started":1}
W20260808 23:00:21.185827 6105 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:21.181438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:35201 (local address 127.23.194.62:43137)
0808 23:00:21.181543 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:21.181545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:21.181553 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:00:21.181576 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:21.181577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:21.181591 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:21.181644 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:21.181976 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:21.182485 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:21.183170 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:21.183374 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:21.183428 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:21.185766 (+ 2338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.3:35201: BlockingRecv error: recv got EOF from 127.23.194.3:35201 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 08 23:00:21 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:21 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:21.930603 6109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:21.926307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.1:38885)
0808 23:00:21.926429 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:21.926439 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:21.926492 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:21.926641 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:21.926641 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:21.926801 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:21.926851 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:21.926852 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:21.927458 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:21.927459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:21.927971 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:21.927973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:21.928329 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:21.928329 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:21.928364 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:21.928826 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:21.928830 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:21.930543 (+ 1713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":66,"threads_started":1}
W20260808 23:00:21.930708 6111 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:21.926397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:38885 (local address 127.23.194.62:43137)
0808 23:00:21.926526 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:21.926528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:21.926537 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:00:21.926561 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:21.926561 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:21.926572 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:21.926624 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:21.926899 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:21.927396 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:21.928061 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:21.928265 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:21.928326 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:21.930632 (+ 2306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.1:38885: BlockingRecv error: recv got EOF from 127.23.194.1:38885 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
Aug 08 23:00:22 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:22 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:22.064159 6113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:22.059910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.2:35897)
0808 23:00:22.060042 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:22.060054 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:22.060116 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:22.060256 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:22.060257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:22.060423 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:22.060470 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:22.060471 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:22.061094 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:22.061095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:22.061674 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:22.061676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:22.062032 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:22.062033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:22.062069 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:22.062494 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:22.062498 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:22.064090 (+ 1592us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
W20260808 23:00:22.064253 6111 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:22.059980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:35897 (local address 127.23.194.62:43137)
0808 23:00:22.060062 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:22.060063 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:22.060100 (+ 37us) server_negotiation.cc:407] Connection header received
0808 23:00:22.060184 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:22.060185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:22.060196 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:22.060242 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:22.060519 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:22.061030 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:22.061772 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:22.061973 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:22.062034 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:22.064195 (+ 2161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.2:35897: BlockingRecv error: recv got EOF from 127.23.194.2:35897 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 23:00:22 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:22 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:22.190621 6115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:22.186207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.3:54639)
0808 23:00:22.186354 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:22.186367 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:22.186433 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:22.186560 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:22.186561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:22.186771 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:22.186820 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:22.186822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:22.187498 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:22.187499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:22.188066 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:22.188068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:22.188394 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:22.188396 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:22.188453 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:22.188879 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:22.188883 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:22.190542 (+ 1659us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":67,"threads_started":1}
W20260808 23:00:22.190697 6111 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:22.186308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:54639 (local address 127.23.194.62:43137)
0808 23:00:22.186368 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:22.186370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:22.186423 (+ 53us) server_negotiation.cc:407] Connection header received
0808 23:00:22.186493 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:22.186494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:22.186508 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:22.186555 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:22.186890 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:22.187416 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:22.188138 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:22.188336 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:22.188389 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:22.190643 (+ 2254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.3:54639: BlockingRecv error: recv got EOF from 127.23.194.3:54639 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 08 23:00:22 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:22 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:22.935328 6116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:22.931080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.1:47817)
0808 23:00:22.931208 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:22.931218 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:22.931267 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:22.931407 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:22.931407 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:22.931572 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:22.931622 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:22.931623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:22.932225 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:22.932226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:22.932803 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:22.932807 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:22.933152 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:22.933153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:22.933207 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:22.933639 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:22.933643 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:22.935260 (+ 1617us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
W20260808 23:00:22.935410 6119 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:22.931171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:47817 (local address 127.23.194.62:43137)
0808 23:00:22.931294 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:22.931296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:22.931304 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:00:22.931326 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:22.931326 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:22.931338 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:22.931397 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:22.931674 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:22.932173 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:22.932876 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:22.933088 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:22.933143 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:22.935349 (+ 2206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.1:47817: BlockingRecv error: recv got EOF from 127.23.194.1:47817 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
Aug 08 23:00:23 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:23 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:23.069291 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:23.064626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.2:54415)
0808 23:00:23.064771 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:23.064783 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:23.064852 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:23.064980 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:23.064981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:23.065174 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:23.065222 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:23.065224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:23.065928 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:23.065930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:23.066455 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:23.066457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:23.066787 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:23.066789 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:23.066845 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:23.067269 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:23.067273 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:23.069196 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
W20260808 23:00:23.069403 6119 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:23.064718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:54415 (local address 127.23.194.62:43137)
0808 23:00:23.064775 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:23.064777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:23.064833 (+ 56us) server_negotiation.cc:407] Connection header received
0808 23:00:23.064906 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:23.064907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:23.064919 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:23.064967 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:23.065300 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:23.065852 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:23.066528 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:23.066725 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:23.066779 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:23.069321 (+ 2542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.2:54415: BlockingRecv error: recv got EOF from 127.23.194.2:54415 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 08 23:00:23 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:23 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:23.195567 6122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:23.191068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.3:48073)
0808 23:00:23.191222 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:23.191234 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:23.191304 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:23.191429 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:23.191430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:23.191612 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:23.191663 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:23.191665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:23.192262 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:23.192263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:23.192786 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:23.192787 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:23.193084 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:23.193085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:23.193126 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:23.193573 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:23.193578 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:23.195496 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":68,"threads_started":1}
W20260808 23:00:23.195676 6119 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:23.191194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:48073 (local address 127.23.194.62:43137)
0808 23:00:23.191249 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:23.191251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:23.191286 (+ 35us) server_negotiation.cc:407] Connection header received
0808 23:00:23.191356 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:23.191356 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:23.191368 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:23.191414 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:23.191715 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:23.192199 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:23.192853 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:23.193028 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:23.193073 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:23.195584 (+ 2511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.3:48073: BlockingRecv error: recv got EOF from 127.23.194.3:48073 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 23:00:23 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:23 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:23.943008 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:23.935787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.1:53857)
0808 23:00:23.935916 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:23.935926 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:23.935975 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:23.936095 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:23.936096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:23.936258 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:23.936306 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:23.936308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:23.936914 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:23.936914 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:23.937478 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:23.937481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:23.937850 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:23.937850 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:23.937889 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:23.941153 (+ 3264us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:23.941158 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:23.942940 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
W20260808 23:00:23.943106 6124 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:23.935867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:53857 (local address 127.23.194.62:43137)
0808 23:00:23.935980 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:23.935982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:23.935992 (+ 10us) server_negotiation.cc:407] Connection header received
0808 23:00:23.936014 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:23.936014 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:23.936026 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:23.936085 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:23.936361 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:23.936857 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:23.937537 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:23.937792 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:23.937844 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:23.943036 (+ 5192us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.1:53857: BlockingRecv error: recv got EOF from 127.23.194.1:53857 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":52,"threads_started":1}
Aug 08 23:00:24 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:24 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:24.074735 6125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:24.069837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.2:35149)
0808 23:00:24.069985 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:24.069995 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:24.070061 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:24.070191 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:24.070191 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:24.070353 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:24.070400 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:24.070401 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:24.071002 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:24.071003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:24.071512 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:24.071513 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:24.071912 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:24.071913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:24.071975 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:24.072584 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:24.072589 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:24.074657 (+ 2068us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
W20260808 23:00:24.074800 6124 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:24.069903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:35149 (local address 127.23.194.62:43137)
0808 23:00:24.069976 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:24.069978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:24.070076 (+ 98us) server_negotiation.cc:407] Connection header received
0808 23:00:24.070107 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:24.070108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:24.070120 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:24.070174 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:24.070449 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:24.070946 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:24.071593 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:24.071835 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:24.071907 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:24.074747 (+ 2840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.2:35149: BlockingRecv error: recv got EOF from 127.23.194.2:35149 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 23:00:24 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:24 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:24.200678 6126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:24.196041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.3:51261)
0808 23:00:24.196195 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:24.196207 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:24.196276 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:24.196429 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:24.196430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:24.196615 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:24.196673 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:24.196675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:24.197318 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:24.197319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:24.197913 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:24.197916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:24.198268 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:24.198269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:24.198326 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:24.198785 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:24.198789 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:24.200600 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":71,"threads_started":1}
W20260808 23:00:24.200755 6124 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:24.196089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:51261 (local address 127.23.194.62:43137)
0808 23:00:24.196151 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:24.196165 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:24.196267 (+ 102us) server_negotiation.cc:407] Connection header received
0808 23:00:24.196349 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:24.196350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:24.196364 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:24.196416 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:24.196745 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:24.197242 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:24.197995 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:24.198188 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:24.198268 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:24.200694 (+ 2426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.3:51261: BlockingRecv error: recv got EOF from 127.23.194.3:51261 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 08 23:00:24 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:24 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:24.948226 6127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:24.943501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.1:53385)
0808 23:00:24.943661 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:24.943672 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:24.943739 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:24.943886 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:24.943887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:24.944097 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:24.944149 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:24.944151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:24.944829 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:24.944831 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:24.945425 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:24.945429 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:24.945740 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:24.945741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:24.945780 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:24.946209 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:24.946214 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:24.948167 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":76,"threads_started":1}
W20260808 23:00:24.948348 6130 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:24.943559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:53385 (local address 127.23.194.62:43137)
0808 23:00:24.943681 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:24.943683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:24.943721 (+ 38us) server_negotiation.cc:407] Connection header received
0808 23:00:24.943786 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:24.943786 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:24.943796 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:24.943856 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:24.944225 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:24.944749 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:24.945490 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:24.945687 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:24.945731 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:24.948273 (+ 2542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.1:53385: BlockingRecv error: recv got EOF from 127.23.194.1:53385 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
Aug 08 23:00:25 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:25 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:25.080827 6131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:25.075409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.2:54411)
0808 23:00:25.075575 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:25.075590 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:25.075671 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:25.075814 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:25.075815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:25.076041 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:25.076101 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:25.076104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:25.076747 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:25.076747 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:25.077586 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:25.077589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:25.078038 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:25.078039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:25.078100 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:25.078602 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:25.078607 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:25.080747 (+ 2140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":98,"threads_started":1}
W20260808 23:00:25.081079 6130 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:25.075540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:54411 (local address 127.23.194.62:43137)
0808 23:00:25.075603 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:25.075605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:25.075665 (+ 60us) server_negotiation.cc:407] Connection header received
0808 23:00:25.075732 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:25.075734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:25.075747 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:25.075799 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:25.076182 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:25.076684 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:25.077672 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:25.077966 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:25.078039 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:25.080847 (+ 2808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.2:54411: BlockingRecv error: recv got EOF from 127.23.194.2:54411 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 23:00:25 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:25 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:25.205933 6133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:25.201125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.3:57119)
0808 23:00:25.201264 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:25.201275 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:25.201330 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:25.201468 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:25.201468 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:25.201628 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:25.201691 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:25.201693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:25.202400 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:25.202401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:25.202935 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:25.202938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:25.203307 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:25.203307 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:25.203354 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:25.203859 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:25.203863 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:25.205871 (+ 2008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
W20260808 23:00:25.206029 6130 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:25.201212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:57119 (local address 127.23.194.62:43137)
0808 23:00:25.201291 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:25.201293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:25.201318 (+ 25us) server_negotiation.cc:407] Connection header received
0808 23:00:25.201399 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:25.201400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:25.201413 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:25.201463 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:25.201793 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:25.202307 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:25.203036 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:25.203229 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:25.203287 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:25.205962 (+ 2675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.3:57119: BlockingRecv error: recv got EOF from 127.23.194.3:57119 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 23:00:25 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:25 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:25.957333 6134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:25.948694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.1:56209)
0808 23:00:25.948819 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:25.948829 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:25.948879 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:25.949248 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:25.949249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:25.949473 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:25.949526 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:25.949528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:25.950239 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:25.950240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:25.950771 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:25.950772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:25.951147 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:25.951147 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:25.951183 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:25.955210 (+ 4027us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:25.955215 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:25.957273 (+ 2058us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
W20260808 23:00:25.957489 6135 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:25.948815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:56209 (local address 127.23.194.62:43137)
0808 23:00:25.948946 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:25.948948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:25.948957 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:00:25.949025 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:25.949025 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:25.949040 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:25.949116 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:25.949609 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:25.950162 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:25.950852 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:25.951081 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:25.951143 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:25.957406 (+ 6263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.1:56209: BlockingRecv error: recv got EOF from 127.23.194.1:56209 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
Aug 08 23:00:26 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:26 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:26.085927 6136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:26.081318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.2:33691)
0808 23:00:26.081489 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:26.081501 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:26.081573 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:26.081706 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:26.081707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:26.081898 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:26.081961 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:26.081963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:26.082628 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:26.082629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:26.083153 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:26.083155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:26.083479 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:26.083481 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:26.083545 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:26.083993 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:26.083997 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:26.085850 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":110,"threads_started":1}
W20260808 23:00:26.085999 6135 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:26.081434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:33691 (local address 127.23.194.62:43137)
0808 23:00:26.081489 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:26.081491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:26.081565 (+ 74us) server_negotiation.cc:407] Connection header received
0808 23:00:26.081630 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:26.081631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:26.081645 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:26.081692 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:26.082044 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:26.082551 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:26.083234 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:26.083421 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:26.083474 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:26.085948 (+ 2474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.2:33691: BlockingRecv error: recv got EOF from 127.23.194.2:33691 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 23:00:26 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:26 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:26.210958 6137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:26.206359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.3:59221)
0808 23:00:26.206523 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:26.206534 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:26.206596 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:26.206718 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:26.206718 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:26.206882 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:26.207010 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:26.207013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:26.207614 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:26.207614 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:26.208123 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:26.208125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:26.208394 (+ 269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:26.208395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:26.208532 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:26.209002 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:26.209008 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:26.210897 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
W20260808 23:00:26.211099 6135 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:26.206438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:59221 (local address 127.23.194.62:43137)
0808 23:00:26.206497 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:26.206500 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:26.206589 (+ 89us) server_negotiation.cc:407] Connection header received
0808 23:00:26.206652 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:26.206653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:26.206665 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:26.206703 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:26.207065 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:26.207560 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:26.208176 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:26.208349 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:26.208392 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:26.211030 (+ 2638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.3:59221: BlockingRecv error: recv got EOF from 127.23.194.3:59221 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 08 23:00:26 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:26 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:26.962167 6138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:26.957772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.1:45819)
0808 23:00:26.957907 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:26.957918 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:26.957986 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:26.958122 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:26.958123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:26.958281 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:26.958329 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:26.958331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:26.958927 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:26.958928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:26.959474 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:26.959476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:26.959809 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:26.959811 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:26.959875 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:26.960325 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:26.960329 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:26.962089 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
W20260808 23:00:26.962260 6139 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:26.957817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:45819 (local address 127.23.194.62:43137)
0808 23:00:26.957940 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:26.957942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:26.957966 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:00:26.958047 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:26.958048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:26.958057 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:26.958108 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:26.958379 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:26.958876 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:26.959531 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:26.959725 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:26.959772 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:26.962189 (+ 2417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.1:45819: BlockingRecv error: recv got EOF from 127.23.194.1:45819 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
Aug 08 23:00:27 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:27 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:27.090831 6143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:27.086381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.2:55129)
0808 23:00:27.086522 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:27.086532 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:27.086596 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:27.086717 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:27.086717 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:27.086896 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:27.086944 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:27.086946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:27.087546 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:27.087547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:27.088099 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:27.088101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:27.088425 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:27.088426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:27.088464 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:27.088866 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:27.088870 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:27.090752 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":72,"threads_started":1}
W20260808 23:00:27.090883 6139 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:27.086457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:55129 (local address 127.23.194.62:43137)
0808 23:00:27.086512 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:27.086514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:27.086589 (+ 75us) server_negotiation.cc:407] Connection header received
0808 23:00:27.086644 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:27.086645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:27.086661 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:27.086706 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:27.086995 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:27.087483 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:27.088189 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:27.088376 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:27.088421 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:27.090830 (+ 2409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.2:55129: BlockingRecv error: recv got EOF from 127.23.194.2:55129 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 08 23:00:27 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:27 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:27.216328 6145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:27.211480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.3:55405)
0808 23:00:27.211648 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:27.211661 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:27.211730 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:27.211876 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:27.211878 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:27.212050 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:27.212104 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:27.212106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:27.212793 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:27.212819 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:27.213565 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:27.213569 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:27.213896 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:27.213897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:27.213938 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:27.214421 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:27.214426 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:27.216267 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":105,"threads_started":1}
W20260808 23:00:27.216465 6139 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:27.211565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:55405 (local address 127.23.194.62:43137)
0808 23:00:27.211628 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:27.211631 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:27.211734 (+ 103us) server_negotiation.cc:407] Connection header received
0808 23:00:27.211791 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:27.211792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:27.211807 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:27.211861 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:27.212181 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:27.212688 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:27.213635 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:27.213836 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:27.213886 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:27.216402 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.3:55405: BlockingRecv error: recv got EOF from 127.23.194.3:55405 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 23:00:27 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:27 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:27.966898 6148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:27.962634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.1:59275)
0808 23:00:27.962763 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:27.962775 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:27.962834 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:27.962970 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:27.962971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:27.963134 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:27.963199 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:27.963201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:27.963809 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:27.963810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:27.964342 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:27.964344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:27.964617 (+ 273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:27.964618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:27.964657 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:27.965150 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:27.965154 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:27.966821 (+ 1667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
W20260808 23:00:27.966984 6151 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:27.962725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:59275 (local address 127.23.194.62:43137)
0808 23:00:27.962859 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:27.962860 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:27.962868 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:00:27.962889 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:27.962890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:27.962902 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:27.962960 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:27.963252 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:27.963750 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:27.964396 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:27.964574 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:27.964615 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:27.966923 (+ 2308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.1:59275: BlockingRecv error: recv got EOF from 127.23.194.1:59275 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
Aug 08 23:00:28 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:28 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:28.097056 6152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:28.091394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.2:53181)
0808 23:00:28.091532 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:28.091543 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:28.091614 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:28.091755 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:28.091756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:28.091931 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:28.091978 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:28.091980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:28.092967 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:28.092969 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:28.093801 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:28.093806 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:28.094276 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:28.094277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:28.094341 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:28.094930 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:28.094935 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:28.096992 (+ 2057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":71,"threads_started":1}
W20260808 23:00:28.097165 6151 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:28.091464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:53181 (local address 127.23.194.62:43137)
0808 23:00:28.091526 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:28.091528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:28.091593 (+ 65us) server_negotiation.cc:407] Connection header received
0808 23:00:28.091685 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:28.091686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:28.091699 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:28.091746 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:28.092047 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:28.092865 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:28.093908 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:28.094177 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:28.094258 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:28.097087 (+ 2829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.2:53181: BlockingRecv error: recv got EOF from 127.23.194.2:53181 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 08 23:00:28 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:28 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:28.221226 6153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:28.216823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.3:46221)
0808 23:00:28.216958 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:28.216969 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:28.217036 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:28.217172 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:28.217173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:28.217358 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:28.217441 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:28.217444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:28.218087 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:28.218089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:28.218616 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:28.218618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:28.218944 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:28.218945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:28.218999 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:28.219442 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:28.219446 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:28.221150 (+ 1704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
W20260808 23:00:28.221320 6151 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:28.216875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:46221 (local address 127.23.194.62:43137)
0808 23:00:28.216919 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:28.216921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:28.217023 (+ 102us) server_negotiation.cc:407] Connection header received
0808 23:00:28.217094 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:28.217095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:28.217108 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:28.217188 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:28.217515 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:28.218011 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:28.218697 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:28.218885 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:28.218940 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:28.221255 (+ 2315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.3:46221: BlockingRecv error: recv got EOF from 127.23.194.3:46221 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 08 23:00:28 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:28 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:28.971808 6154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:28.967425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.1:47895)
0808 23:00:28.967552 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:28.967562 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:28.967609 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:28.967742 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:28.967743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:28.967913 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:28.967964 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:28.967966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:28.968584 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:28.968585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:28.969217 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:28.969221 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:28.969640 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:28.969641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:28.969692 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:28.970108 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:28.970112 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:28.971749 (+ 1637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":57,"threads_started":1}
W20260808 23:00:28.971900 6155 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:28.967483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:47895 (local address 127.23.194.62:43137)
0808 23:00:28.967595 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:28.967597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:28.967606 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:00:28.967663 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:28.967663 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:28.967674 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:28.967756 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:28.968020 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:28.968516 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:28.969295 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:28.969571 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:28.969621 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:28.971831 (+ 2210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.1:47895: BlockingRecv error: recv got EOF from 127.23.194.1:47895 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":57,"threads_started":1}
Aug 08 23:00:29 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:29 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:29.102252 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:29.097560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.2:57289)
0808 23:00:29.097729 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:29.097741 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:29.097809 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:29.097960 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:29.097961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:29.098134 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:29.098185 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:29.098186 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:29.099037 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:29.099038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:29.099563 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:29.099565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:29.099906 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:29.099907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:29.099963 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:29.100390 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:29.100395 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:29.102174 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":70,"threads_started":1}
W20260808 23:00:29.102320 6155 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:29.097699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:57289 (local address 127.23.194.62:43137)
0808 23:00:29.097744 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:29.097746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:29.097861 (+ 115us) server_negotiation.cc:407] Connection header received
0808 23:00:29.097886 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:29.097886 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:29.097897 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:29.097939 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:29.098256 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:29.098959 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:29.099647 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:29.099846 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:29.099900 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:29.102268 (+ 2368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.2:57289: BlockingRecv error: recv got EOF from 127.23.194.2:57289 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 23:00:29 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:29 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:29.226266 6157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:29.221675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.3:35781)
0808 23:00:29.221827 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:29.221837 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:29.221898 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:29.222028 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:29.222028 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:29.222213 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:29.222259 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:29.222261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:29.222882 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:29.222889 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:29.223448 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:29.223451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:29.223785 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:29.223786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:29.223833 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:29.224264 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:29.224268 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:29.226180 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":89,"threads_started":1}
W20260808 23:00:29.226313 6155 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:29.221750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:35781 (local address 127.23.194.62:43137)
0808 23:00:29.221825 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:29.221827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:29.221920 (+ 93us) server_negotiation.cc:407] Connection header received
0808 23:00:29.221949 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:29.221949 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:29.221964 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:29.222017 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:29.222313 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:29.222795 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:29.223532 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:29.223716 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:29.223769 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:29.226261 (+ 2492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.3:35781: BlockingRecv error: recv got EOF from 127.23.194.3:35781 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 23:00:29 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:29 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:29.977008 6158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:29.972316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.1:54287)
0808 23:00:29.972468 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:29.972479 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:29.972544 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:29.972690 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:29.972691 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:29.972893 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:29.972954 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:29.972957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:29.973799 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:29.973801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:29.974338 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:29.974340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:29.974709 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:29.974710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:29.974766 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:29.975210 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:29.975214 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:29.976940 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":70,"threads_started":1}
W20260808 23:00:29.977156 6159 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:29.972399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:54287 (local address 127.23.194.62:43137)
0808 23:00:29.972521 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:29.972523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:29.972532 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:00:29.972600 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:29.972600 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:29.972612 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:29.972682 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:29.973079 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:29.973718 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:29.974427 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:29.974634 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:29.974690 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:29.977075 (+ 2385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.1:54287: BlockingRecv error: recv got EOF from 127.23.194.1:54287 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":52,"threads_started":1}
Aug 08 23:00:30 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:30 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:30.107074 6161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:30.102753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.2:49875)
0808 23:00:30.102877 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:30.102888 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:30.102953 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:30.103100 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:30.103101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:30.103266 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:30.103315 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:30.103317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:30.103921 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:30.103922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:30.104455 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:30.104457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:30.104784 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:30.104785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:30.104823 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:30.105269 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:30.105274 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:30.107014 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":74,"threads_started":1}
W20260808 23:00:30.107144 6159 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:30.102821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:49875 (local address 127.23.194.62:43137)
0808 23:00:30.102874 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:30.102876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:30.102936 (+ 60us) server_negotiation.cc:407] Connection header received
0808 23:00:30.103030 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:30.103031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:30.103041 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:30.103094 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:30.103368 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:30.103862 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:30.104532 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:30.104725 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:30.104777 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:30.107095 (+ 2318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.2:49875: BlockingRecv error: recv got EOF from 127.23.194.2:49875 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 08 23:00:30 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:30 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:30.230999 6162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:30.226759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.3:59201)
0808 23:00:30.226925 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:30.226936 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:30.226989 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:30.227122 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:30.227123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:30.227289 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:30.227337 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:30.227339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:30.227924 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:30.227925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:30.228435 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:30.228436 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:30.228721 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:30.228722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:30.228764 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:30.229232 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:30.229236 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:30.230937 (+ 1701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":81,"threads_started":1}
W20260808 23:00:30.231084 6159 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:30.226912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:59201 (local address 127.23.194.62:43137)
0808 23:00:30.226970 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:30.226972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:30.226979 (+ 7us) server_negotiation.cc:407] Connection header received
0808 23:00:30.227037 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:30.227038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:30.227056 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:30.227106 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:30.227390 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:30.227870 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:30.228494 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:30.228665 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:30.228708 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:30.231025 (+ 2317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.3:59201: BlockingRecv error: recv got EOF from 127.23.194.3:59201 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 08 23:00:30 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:30 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:30.985884 6164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:30.977469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.1:38731)
0808 23:00:30.977617 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:30.977627 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:30.977703 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:30.977831 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:30.977832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:30.978002 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:30.978048 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:30.978050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:30.978683 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:30.978684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:30.979196 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:30.979197 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:30.979542 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:30.979543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:30.979578 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:30.983822 (+ 4244us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:30.983826 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:30.985808 (+ 1982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
W20260808 23:00:30.985962 6165 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:30.977548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:38731 (local address 127.23.194.62:43137)
0808 23:00:30.977654 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:30.977656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:30.977689 (+ 33us) server_negotiation.cc:407] Connection header received
0808 23:00:30.977752 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:30.977753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:30.977763 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:30.977822 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:30.978107 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:30.978621 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:30.979268 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:30.979468 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:30.979542 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:30.985900 (+ 6358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.1:38731: BlockingRecv error: recv got EOF from 127.23.194.1:38731 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":53,"threads_started":1}
Aug 08 23:00:31 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:31 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:31.111840 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:31.107549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.2:38289)
0808 23:00:31.107698 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:31.107708 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:31.107769 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:31.107885 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:31.107886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:31.108085 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:31.108139 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:31.108141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:31.108743 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:31.108743 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:31.109255 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:31.109256 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:31.109554 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:31.109555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:31.109597 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:31.110005 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:31.110009 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:31.111745 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":71,"threads_started":1}
W20260808 23:00:31.111903 6165 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:31.107614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:38289 (local address 127.23.194.62:43137)
0808 23:00:31.107681 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:31.107683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:31.107778 (+ 95us) server_negotiation.cc:407] Connection header received
0808 23:00:31.107810 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:31.107810 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:31.107821 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:31.107875 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:31.108192 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:31.108685 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:31.109312 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:31.109506 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:31.109553 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:31.111851 (+ 2298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.2:38289: BlockingRecv error: recv got EOF from 127.23.194.2:38289 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 23:00:31 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:31 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:31.235538 6168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:31.231422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.3:45325)
0808 23:00:31.231551 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:31.231562 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:31.231632 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:31.231768 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:31.231769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:31.231939 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:31.231987 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:31.231988 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:31.232603 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:31.232605 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:31.233128 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:31.233130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:31.233433 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:31.233434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:31.233488 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:31.233893 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:31.233898 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:31.235457 (+ 1559us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
W20260808 23:00:31.235594 6165 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:31.231471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:45325 (local address 127.23.194.62:43137)
0808 23:00:31.231552 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:31.231554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:31.231617 (+ 63us) server_negotiation.cc:407] Connection header received
0808 23:00:31.231694 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:31.231695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:31.231708 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:31.231756 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:31.232035 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:31.232522 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:31.233191 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:31.233361 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:31.233429 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:31.235543 (+ 2114us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.3:45325: BlockingRecv error: recv got EOF from 127.23.194.3:45325 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 23:00:31 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:31 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:31.990922 6169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:31.986350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.1:55257)
0808 23:00:31.986486 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:31.986497 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:31.986548 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:31.986694 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:31.986695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:31.986870 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:31.986916 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:31.986918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:31.987598 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:31.987598 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:31.988185 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:31.988188 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:31.988634 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:31.988635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:31.988694 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:31.989146 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:31.989152 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:31.990860 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":61,"threads_started":1}
W20260808 23:00:31.991026 6170 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:31.986436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:55257 (local address 127.23.194.62:43137)
0808 23:00:31.986551 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:31.986553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:31.986562 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:00:31.986614 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:31.986615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:31.986626 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:31.986677 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:31.986968 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:31.987529 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:31.988271 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:31.988547 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:31.988609 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:31.990957 (+ 2348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.1:55257: BlockingRecv error: recv got EOF from 127.23.194.1:55257 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":60,"threads_started":1}
Aug 08 23:00:32 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:32 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:32.116811 6171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:32.112295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.2:44607)
0808 23:00:32.112448 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:32.112458 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:32.112516 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:32.112638 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:32.112639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:32.112831 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:32.112889 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:32.112891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:32.113527 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:32.113528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:32.114085 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:32.114087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:32.114381 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:32.114382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:32.114419 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:32.114844 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:32.114848 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:32.116735 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
W20260808 23:00:32.116880 6170 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:32.112366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:44607 (local address 127.23.194.62:43137)
0808 23:00:32.112410 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:32.112412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:32.112502 (+ 90us) server_negotiation.cc:407] Connection header received
0808 23:00:32.112562 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:32.112563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:32.112573 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:32.112628 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:32.112943 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:32.113457 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:32.114147 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:32.114332 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:32.114379 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:32.116828 (+ 2449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.2:44607: BlockingRecv error: recv got EOF from 127.23.194.2:44607 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 08 23:00:32 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:32 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:32.240281 6172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:32.235987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.3:37011)
0808 23:00:32.236130 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:32.236141 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:32.236209 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:32.236325 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:32.236325 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:32.236498 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:32.236547 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:32.236548 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:32.237173 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:32.237174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:32.237743 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:32.237745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:32.238025 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:32.238026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:32.238063 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:32.238521 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:32.238526 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:32.240213 (+ 1687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
W20260808 23:00:32.240376 6170 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:32.236059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:37011 (local address 127.23.194.62:43137)
0808 23:00:32.236117 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:32.236119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:32.236203 (+ 84us) server_negotiation.cc:407] Connection header received
0808 23:00:32.236263 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:32.236264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:32.236276 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:32.236312 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:32.236596 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:32.237091 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:32.237804 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:32.237978 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:32.238021 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:32.240312 (+ 2291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.3:37011: BlockingRecv error: recv got EOF from 127.23.194.3:37011 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 08 23:00:32 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:32 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:32.995802 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:32.991391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.1:58761)
0808 23:00:32.991513 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:32.991524 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:32.991570 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:32.991747 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:32.991748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:32.991913 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:32.991962 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:32.991963 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:32.992575 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:32.992575 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:32.993106 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:32.993108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:32.993443 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:32.993444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:32.993487 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:32.993884 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:32.993888 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:32.995710 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
W20260808 23:00:32.995882 6176 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:32.991500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:58761 (local address 127.23.194.62:43137)
0808 23:00:32.991620 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:32.991623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:32.991631 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:00:32.991669 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:32.991670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:32.991680 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:32.991736 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:32.992014 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:32.992519 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:32.993169 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:32.993354 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:32.993428 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:32.995804 (+ 2376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.1:58761: BlockingRecv error: recv got EOF from 127.23.194.1:58761 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":56,"threads_started":1}
Aug 08 23:00:33 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:33 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:33.121742 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:33.117267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.2:54215)
0808 23:00:33.117436 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:33.117447 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:33.117504 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:33.117628 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:33.117629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:33.117803 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:33.117850 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:33.117852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:33.118447 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:33.118448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:33.118998 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:33.119001 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:33.119458 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:33.119459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:33.119523 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:33.119988 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:33.119994 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:33.121659 (+ 1665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":91,"threads_started":1}
W20260808 23:00:33.121802 6176 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:33.117412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:54215 (local address 127.23.194.62:43137)
0808 23:00:33.117459 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:33.117461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:33.117489 (+ 28us) server_negotiation.cc:407] Connection header received
0808 23:00:33.117554 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:33.117555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:33.117566 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:33.117618 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:33.117899 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:33.118393 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:33.119073 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:33.119373 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:33.119437 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:33.121756 (+ 2319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.2:54215: BlockingRecv error: recv got EOF from 127.23.194.2:54215 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 08 23:00:33 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:33 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:33.245502 6180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:33.240735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.3:42879)
0808 23:00:33.240912 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:33.240924 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:33.240993 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:33.241137 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:33.241138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:33.241318 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:33.241364 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:33.241366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:33.242191 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:33.242192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:33.242771 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:33.242774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:33.243278 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:33.243279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:33.243349 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:33.243823 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:33.243829 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:33.245423 (+ 1594us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":94,"threads_started":1}
W20260808 23:00:33.245554 6176 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:33.240811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:42879 (local address 127.23.194.62:43137)
0808 23:00:33.240882 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:33.240885 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:33.240986 (+ 101us) server_negotiation.cc:407] Connection header received
0808 23:00:33.241060 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:33.241061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:33.241075 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:33.241116 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:33.241477 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:33.242114 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:33.242857 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:33.243204 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:33.243270 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:33.245500 (+ 2230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.3:42879: BlockingRecv error: recv got EOF from 127.23.194.3:42879 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 23:00:33 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:33 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:34.000509 6181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:33.996243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.1:52559)
0808 23:00:33.996376 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:33.996387 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:33.996440 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:33.996556 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:33.996557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:33.996734 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:33.996781 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:33.996783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:33.997441 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:33.997442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:33.997967 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:33.997969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:33.998267 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:33.998268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:33.998307 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:33.998774 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:33.998779 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:34.000447 (+ 1668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
W20260808 23:00:34.000627 6184 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:33.996307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:52559 (local address 127.23.194.62:43137)
0808 23:00:33.996445 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:33.996447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:33.996457 (+ 10us) server_negotiation.cc:407] Connection header received
0808 23:00:33.996481 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:33.996482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:33.996491 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:33.996546 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:33.996849 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:33.997351 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:33.998033 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:33.998220 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:33.998262 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:34.000555 (+ 2293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.1:52559: BlockingRecv error: recv got EOF from 127.23.194.1:52559 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
Aug 08 23:00:34 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:34 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:34.126480 6187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:34.122160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.2:38033)
0808 23:00:34.122288 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:34.122300 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:34.122376 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:34.122530 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:34.122531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:34.122737 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:34.122786 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:34.122788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:34.123404 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:34.123405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:34.123925 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:34.123926 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:34.124215 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:34.124216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:34.124259 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:34.124709 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:34.124715 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:34.126420 (+ 1705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
W20260808 23:00:34.126569 6184 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:34.122238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:38033 (local address 127.23.194.62:43137)
0808 23:00:34.122292 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:34.122294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:34.122357 (+ 63us) server_negotiation.cc:407] Connection header received
0808 23:00:34.122444 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:34.122445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:34.122458 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:34.122504 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:34.122848 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:34.123347 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:34.123978 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:34.124162 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:34.124204 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:34.126509 (+ 2305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.2:38033: BlockingRecv error: recv got EOF from 127.23.194.2:38033 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 08 23:00:34 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:34 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.3@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:34.250296 6188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:34.246011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.3:49085)
0808 23:00:34.246146 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:34.246188 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:34.246246 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:34.246360 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:34.246360 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:34.246530 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:34.246579 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:34.246580 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:34.247188 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:34.247189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:34.247763 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:34.247765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:34.248077 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:34.248078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:34.248113 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:34.248550 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:34.248554 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:34.250225 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":61,"threads_started":1}
W20260808 23:00:34.250378 6184 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:34.246097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:49085 (local address 127.23.194.62:43137)
0808 23:00:34.246150 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:34.246152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:34.246229 (+ 77us) server_negotiation.cc:407] Connection header received
0808 23:00:34.246287 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:34.246288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:34.246299 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:34.246349 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:34.246626 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:34.247118 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:34.247829 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:34.248023 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:34.248074 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:34.250316 (+ 2242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.3:49085: BlockingRecv error: recv got EOF from 127.23.194.3:49085 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 08 23:00:35 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:35 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.1@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:35.005685 6189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:35.001017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.1:58363)
0808 23:00:35.001139 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:35.001151 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:35.001200 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:35.001349 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:35.001350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:35.001530 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:35.001580 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:35.001581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.002227 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.002227 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:35.002798 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:35.002801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.003186 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.003186 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:35.003225 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:35.003671 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:35.003676 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:35.005607 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
W20260808 23:00:35.005769 6192 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:35.001094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:58363 (local address 127.23.194.62:43137)
0808 23:00:35.001232 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:35.001234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:35.001242 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:00:35.001265 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:35.001266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:35.001277 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:35.001340 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:35.001639 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.002169 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.002895 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.003117 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.003176 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:35.005699 (+ 2523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.1:58363: BlockingRecv error: recv got EOF from 127.23.194.1:58363 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
Aug 08 23:00:35 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:00:35 dist-test-slave-f0tn krb5kdc[4554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.194.2@KRBTEST.COM for oryx/127.23.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:00:35.131108 6193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:35.126955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:43137 (local address 127.23.194.2:42103)
0808 23:00:35.127096 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:35.127107 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:35.127177 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:35.127290 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:35.127290 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:35.127444 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:35.127491 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:35.127493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.128072 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.128072 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:35.128596 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:35.128598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.128879 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.128880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:35.128917 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:35.129318 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:35.129322 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:35.131050 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.194.62:43137: Server oryx/127.23.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
W20260808 23:00:35.131204 6192 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:35.127025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:42103 (local address 127.23.194.62:43137)
0808 23:00:35.127088 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:35.127090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:35.127182 (+ 92us) server_negotiation.cc:407] Connection header received
0808 23:00:35.127220 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:35.127221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:35.127231 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:35.127280 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:35.127541 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.128021 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.128657 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.128832 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.128875 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:35.131141 (+ 2266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.194.2:42103: BlockingRecv error: recv got EOF from 127.23.194.2:42103 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 23:00:35.174806 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 5468
I20260808 23:00:35.189208 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 5696
I20260808 23:00:35.195241 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 5897
I20260808 23:00:35.204835 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 5364
2026-08-08T23:00:35Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17475 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T23:00:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:00:35Z Disabled control of system clock
I20260808 23:00:35.237542 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:34357
--webserver_interface=127.23.194.62
--webserver_port=0
--builtin_ntp_servers=127.23.194.20:44621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:34357
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 23:00:35.320446 6199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:35.320648 6199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:35.320677 6199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:35.321940 6199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:00:35.321983 6199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:35.321995 6199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:35.322006 6199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:00:35.322017 6199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:00:35.323349 6199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:44621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:34357
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:34357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:35.323547 6199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:35.323838 6199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:35.329265 6214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:35.329320 6199 server_base.cc:1034] running on GCE node
W20260808 23:00:35.329360 6208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:35.329414 6207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:35.329823 6199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:35.330109 6199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:35.331264 6199 hybrid_clock.cc:648] HybridClock initialized: now 1786230035331243 us; error 33 us; skew 500 ppm
I20260808 23:00:35.332792 6199 webserver.cc:460] Webserver started at http://127.23.194.62:36233/ using document root <none> and password file <none>
I20260808 23:00:35.333015 6199 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:35.333065 6199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:35.333214 6199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:35.334309 6199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "cc467851cfdb47e3adb8efb9a7399748"
format_stamp: "Formatted at 2026-08-08 23:00:35 on dist-test-slave-f0tn"
server_key: "6e585c38df65b0706531e714a66caced"
server_key_iv: "2e81bc02933634f2c9c85696f3b72148"
server_key_version: "encryptionkey@0"
I20260808 23:00:35.334643 6199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "cc467851cfdb47e3adb8efb9a7399748"
format_stamp: "Formatted at 2026-08-08 23:00:35 on dist-test-slave-f0tn"
server_key: "6e585c38df65b0706531e714a66caced"
server_key_iv: "2e81bc02933634f2c9c85696f3b72148"
server_key_version: "encryptionkey@0"
I20260808 23:00:35.335846 6199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 23:00:35.336499 6227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:35.336658 6199 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:35.336742 6199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "cc467851cfdb47e3adb8efb9a7399748"
format_stamp: "Formatted at 2026-08-08 23:00:35 on dist-test-slave-f0tn"
server_key: "6e585c38df65b0706531e714a66caced"
server_key_iv: "2e81bc02933634f2c9c85696f3b72148"
server_key_version: "encryptionkey@0"
I20260808 23:00:35.336808 6199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:35.345770 6199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:35.346028 6199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:35.346155 6199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:35.349787 6199 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:34357
I20260808 23:00:35.349817 6315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:34357 every 8 connection(s)
I20260808 23:00:35.350142 6199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:00:35.350791 6316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:35.352721 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 6199
I20260808 23:00:35.352803 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:00:35.352984 24328 external_mini_cluster.cc:1428] Setting key 44727612f54f9a5a4f1bcd3e8c4686c7
I20260808 23:00:35.353664 6316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748: Bootstrap starting.
I20260808 23:00:35.354380 6316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:35.354719 6316 log.cc:824] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:35.355448 6316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748: No bootstrap required, opened a new log
I20260808 23:00:35.357642 6316 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc467851cfdb47e3adb8efb9a7399748" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34357 } }
I20260808 23:00:35.357796 6316 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:35.357831 6316 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc467851cfdb47e3adb8efb9a7399748, State: Initialized, Role: FOLLOWER
I20260808 23:00:35.357990 6316 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc467851cfdb47e3adb8efb9a7399748" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34357 } }
I20260808 23:00:35.358094 6316 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:00:35.358131 6316 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:00:35.358181 6316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:35.358810 6321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:35.353525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49830 (local address 127.23.194.62:34357)
0808 23:00:35.353728 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:35.353736 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:35.353768 (+ 32us) server_negotiation.cc:407] Connection header received
0808 23:00:35.354315 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:35.354323 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:35.354404 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:35.354566 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:35.354723 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.355931 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.356686 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.357314 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.357481 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:35.358153 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:35.358157 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:35.358158 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:35.358343 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:35.358375 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:35.358386 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:35.358435 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:35.358541 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 23:00:35.358603 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
I20260808 23:00:35.358819 6316 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc467851cfdb47e3adb8efb9a7399748" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34357 } }
I20260808 23:00:35.358937 6316 leader_election.cc:302] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc467851cfdb47e3adb8efb9a7399748; no voters:
I20260808 23:00:35.359094 6316 leader_election.cc:288] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:00:35.359133 6323 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:35.359297 6323 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748 [term 1 LEADER]: Becoming Leader. State: Replica: cc467851cfdb47e3adb8efb9a7399748, State: Running, Role: LEADER
I20260808 23:00:35.359392 6316 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:00:35.359413 6323 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc467851cfdb47e3adb8efb9a7399748" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34357 } }
I20260808 23:00:35.359854 6323 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc467851cfdb47e3adb8efb9a7399748. Latest consensus state: current_term: 1 leader_uuid: "cc467851cfdb47e3adb8efb9a7399748" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc467851cfdb47e3adb8efb9a7399748" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34357 } } }
I20260808 23:00:35.360009 6324 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc467851cfdb47e3adb8efb9a7399748" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc467851cfdb47e3adb8efb9a7399748" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34357 } } }
I20260808 23:00:35.360078 6323 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:35.360181 6324 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:35.360570 6338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:00:35.361052 6338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:00:35.362439 6338 catalog_manager.cc:1347] Generated new cluster ID: 87c67b17bf13478ba67af9a88adff4a3
I20260808 23:00:35.362497 6338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:00:35.376278 6338 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:00:35.377007 6338 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:00:35.389631 6338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc467851cfdb47e3adb8efb9a7399748: Generated new TSK 0
I20260808 23:00:35.389871 6338 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:00:35.394402 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34357
--builtin_ntp_servers=127.23.194.20:44621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 23:00:35.476645 6356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:35.476895 6356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:35.476938 6356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:35.478168 6356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:35.478215 6356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:35.478253 6356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:00:35.479691 6356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:44621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:35.479954 6356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:35.480262 6356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:35.485728 6369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:35.485751 6366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:35.485756 6364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:35.485986 6356 server_base.cc:1034] running on GCE node
I20260808 23:00:35.486251 6356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:35.486507 6356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:35.487648 6356 hybrid_clock.cc:648] HybridClock initialized: now 1786230035487631 us; error 33 us; skew 500 ppm
I20260808 23:00:35.488943 6356 webserver.cc:460] Webserver started at http://127.23.194.1:36197/ using document root <none> and password file <none>
I20260808 23:00:35.489184 6356 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:35.489235 6356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:35.489367 6356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:35.490485 6356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "eb14d6be227a468cbd7c00460bed3602"
format_stamp: "Formatted at 2026-08-08 23:00:35 on dist-test-slave-f0tn"
server_key: "89813f4c93575c609e686fb605324790"
server_key_iv: "a59b7e636890914e24fe374e11baa6b9"
server_key_version: "encryptionkey@0"
I20260808 23:00:35.490837 6356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "eb14d6be227a468cbd7c00460bed3602"
format_stamp: "Formatted at 2026-08-08 23:00:35 on dist-test-slave-f0tn"
server_key: "89813f4c93575c609e686fb605324790"
server_key_iv: "a59b7e636890914e24fe374e11baa6b9"
server_key_version: "encryptionkey@0"
I20260808 23:00:35.492149 6356 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 23:00:35.492916 6380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:35.493072 6356 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:00:35.493158 6356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "eb14d6be227a468cbd7c00460bed3602"
format_stamp: "Formatted at 2026-08-08 23:00:35 on dist-test-slave-f0tn"
server_key: "89813f4c93575c609e686fb605324790"
server_key_iv: "a59b7e636890914e24fe374e11baa6b9"
server_key_version: "encryptionkey@0"
I20260808 23:00:35.493232 6356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:35.506824 6356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:35.507078 6356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:35.507200 6356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:35.507392 6356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:35.507699 6356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:35.507740 6356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:35.507766 6356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:35.507793 6356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:35.513254 6356 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:36943
I20260808 23:00:35.513322 6563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:36943 every 8 connection(s)
I20260808 23:00:35.513616 6356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:00:35.518201 6321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:35.514290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:58317 (local address 127.23.194.62:34357)
0808 23:00:35.514352 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:35.514354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:35.514565 (+ 211us) server_negotiation.cc:407] Connection header received
0808 23:00:35.515024 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:35.515025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:35.515038 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:35.515089 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:35.515690 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.516178 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.517074 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.517262 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.517305 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:35.517988 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:35.517989 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:35.517990 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:35.517994 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:35.518008 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:35.518012 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:35.518051 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:35.518110 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 23:00:35.518129 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 23:00:35.518362 6568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:35.514270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34357 (local address 127.23.194.1:58317)
0808 23:00:35.514455 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:35.514486 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:35.514949 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:35.515139 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:35.515142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:35.515150 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:35.515615 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:35.515619 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.516242 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.516242 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:35.517012 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:35.517014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.517317 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.517318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:35.517428 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:35.517868 (+ 440us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:00:35.517868 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:35.517895 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:00:35.517917 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:35.518057 (+ 140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:35.518059 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:35.518062 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:00:35.518106 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 23:00:35.518183 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260808 23:00:35.518838 6566 heartbeater.cc:347] Connected to a master server at 127.23.194.62:34357
I20260808 23:00:35.518922 6566 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:35.519094 6566 heartbeater.cc:511] Master 127.23.194.62:34357 requested a full tablet report, sending...
I20260808 23:00:35.519562 6252 ts_manager.cc:194] Registered new tserver with Master: eb14d6be227a468cbd7c00460bed3602 (127.23.194.1:36943)
I20260808 23:00:35.519975 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 6356
I20260808 23:00:35.520058 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:00:35.520253 24328 external_mini_cluster.cc:1428] Setting key a3ab1566b97d764ab442459c2f186dba
I20260808 23:00:35.520392 6252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.194.1:58317
I20260808 23:00:35.521298 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34357
--builtin_ntp_servers=127.23.194.20:44621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 23:00:35.601847 6569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:35.602051 6569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:35.602072 6569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:35.603200 6569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:35.603240 6569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:35.603267 6569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:00:35.604591 6569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:44621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:35.604818 6569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:35.605083 6569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:35.609933 6579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:35.609941 6578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:35.610028 6569 server_base.cc:1034] running on GCE node
W20260808 23:00:35.609993 6583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:35.610374 6569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:35.610574 6569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:35.611703 6569 hybrid_clock.cc:648] HybridClock initialized: now 1786230035611689 us; error 28 us; skew 500 ppm
I20260808 23:00:35.612916 6569 webserver.cc:460] Webserver started at http://127.23.194.2:34987/ using document root <none> and password file <none>
I20260808 23:00:35.613125 6569 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:35.613183 6569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:35.613307 6569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:35.614583 6569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "343b098e614844af8e0c4646d5878b04"
format_stamp: "Formatted at 2026-08-08 23:00:35 on dist-test-slave-f0tn"
server_key: "9896f674d0510c38eef4442d8eae8c5e"
server_key_iv: "33c24a1f26eea70247e629182a0dbe14"
server_key_version: "encryptionkey@0"
I20260808 23:00:35.614918 6569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "343b098e614844af8e0c4646d5878b04"
format_stamp: "Formatted at 2026-08-08 23:00:35 on dist-test-slave-f0tn"
server_key: "9896f674d0510c38eef4442d8eae8c5e"
server_key_iv: "33c24a1f26eea70247e629182a0dbe14"
server_key_version: "encryptionkey@0"
I20260808 23:00:35.616113 6569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:00:35.616729 6590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:35.616863 6569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:35.616947 6569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "343b098e614844af8e0c4646d5878b04"
format_stamp: "Formatted at 2026-08-08 23:00:35 on dist-test-slave-f0tn"
server_key: "9896f674d0510c38eef4442d8eae8c5e"
server_key_iv: "33c24a1f26eea70247e629182a0dbe14"
server_key_version: "encryptionkey@0"
I20260808 23:00:35.617013 6569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:35.668008 6569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:35.668334 6569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:35.668471 6569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:35.668722 6569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:35.669121 6569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:35.669162 6569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:35.669193 6569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:35.669209 6569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:35.674955 6569 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:33725
I20260808 23:00:35.675014 6788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:33725 every 8 connection(s)
I20260808 23:00:35.675434 6569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:00:35.676556 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 6569
I20260808 23:00:35.676661 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:00:35.676877 24328 external_mini_cluster.cc:1428] Setting key b2bcdc5efa7b2612c4de6e07a484a674
I20260808 23:00:35.677924 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34357
--builtin_ntp_servers=127.23.194.20:44621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 23:00:35.681200 6321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:35.676172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:45811 (local address 127.23.194.62:34357)
0808 23:00:35.676225 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:35.676226 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:35.676438 (+ 212us) server_negotiation.cc:407] Connection header received
0808 23:00:35.677120 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:35.677121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:35.677133 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:35.677173 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:35.678066 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.678591 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.679862 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.680073 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.680130 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:35.680945 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:35.680946 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:35.680946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:35.680951 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:35.680968 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:35.680972 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:35.681018 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:35.681091 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 23:00:35.681118 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 23:00:35.681437 6791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:35.676139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34357 (local address 127.23.194.2:45811)
0808 23:00:35.676313 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:35.676347 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:35.677015 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:35.677254 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:35.677258 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:35.677269 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:35.677935 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:35.677943 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.678687 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.678689 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:35.679783 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:35.679786 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.680155 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.680156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:35.680292 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:35.680812 (+ 520us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:00:35.680813 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:35.680844 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:00:35.680867 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:35.681024 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:35.681030 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:35.681033 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:00:35.681095 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 23:00:35.681202 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
I20260808 23:00:35.681950 6789 heartbeater.cc:347] Connected to a master server at 127.23.194.62:34357
I20260808 23:00:35.682050 6789 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:35.682250 6789 heartbeater.cc:511] Master 127.23.194.62:34357 requested a full tablet report, sending...
I20260808 23:00:35.682611 6251 ts_manager.cc:194] Registered new tserver with Master: 343b098e614844af8e0c4646d5878b04 (127.23.194.2:33725)
I20260808 23:00:35.682981 6251 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.194.2:45811
W20260808 23:00:35.761523 6792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:35.761765 6792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:35.761797 6792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:35.762954 6792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:35.763031 6792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:35.763070 6792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:00:35.764405 6792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:44621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:35.764644 6792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:35.764928 6792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:35.769824 6804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:35.769824 6807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:35.769918 6792 server_base.cc:1034] running on GCE node
W20260808 23:00:35.769853 6805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:35.770303 6792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:35.770540 6792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:35.771672 6792 hybrid_clock.cc:648] HybridClock initialized: now 1786230035771657 us; error 30 us; skew 500 ppm
I20260808 23:00:35.772902 6792 webserver.cc:460] Webserver started at http://127.23.194.3:38115/ using document root <none> and password file <none>
I20260808 23:00:35.773118 6792 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:35.773166 6792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:35.773273 6792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:35.774355 6792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "ba35f2602c6546a9ac0ef11bf4f17046"
format_stamp: "Formatted at 2026-08-08 23:00:35 on dist-test-slave-f0tn"
server_key: "a4498bd203dd14598435ed9df7df2b22"
server_key_iv: "1e97ee8c2fce1e039b064b8a47803f4d"
server_key_version: "encryptionkey@0"
I20260808 23:00:35.774686 6792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "ba35f2602c6546a9ac0ef11bf4f17046"
format_stamp: "Formatted at 2026-08-08 23:00:35 on dist-test-slave-f0tn"
server_key: "a4498bd203dd14598435ed9df7df2b22"
server_key_iv: "1e97ee8c2fce1e039b064b8a47803f4d"
server_key_version: "encryptionkey@0"
I20260808 23:00:35.775905 6792 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:00:35.776543 6818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:35.776685 6792 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:35.776768 6792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "ba35f2602c6546a9ac0ef11bf4f17046"
format_stamp: "Formatted at 2026-08-08 23:00:35 on dist-test-slave-f0tn"
server_key: "a4498bd203dd14598435ed9df7df2b22"
server_key_iv: "1e97ee8c2fce1e039b064b8a47803f4d"
server_key_version: "encryptionkey@0"
I20260808 23:00:35.776829 6792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:35.785240 6792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:35.785521 6792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:35.785643 6792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:35.785836 6792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:35.786137 6792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:35.786171 6792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:35.786198 6792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:35.786222 6792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:35.791546 6792 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:44085
I20260808 23:00:35.791596 6998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:44085 every 8 connection(s)
I20260808 23:00:35.791890 6792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 23:00:35.793205 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 6792
I20260808 23:00:35.793292 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:00:35.793527 24328 external_mini_cluster.cc:1428] Setting key 8e63a1f829f73e73ae1fc7b7ddf50108
I20260808 23:00:35.796993 6321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:35.792611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:57921 (local address 127.23.194.62:34357)
0808 23:00:35.792657 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:35.792659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:35.792849 (+ 190us) server_negotiation.cc:407] Connection header received
0808 23:00:35.793351 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:35.793352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:35.793365 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:35.793482 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:35.794331 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.794846 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.795823 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.796005 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.796049 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:35.796798 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:35.796799 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:35.796800 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:35.796804 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:35.796816 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:35.796819 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:35.796853 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:35.796913 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 23:00:35.796933 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 23:00:35.797245 7001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:35.792562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34357 (local address 127.23.194.3:57921)
0808 23:00:35.792736 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:35.792768 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:35.793253 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:35.793561 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:35.793566 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:35.793576 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:35.794239 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:35.794246 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.794927 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.794928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:35.795748 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:35.795750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:35.796066 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:35.796067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:35.796192 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:35.796679 (+ 487us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:00:35.796680 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:35.796708 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:00:35.796729 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:35.796863 (+ 134us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:35.796865 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:35.796867 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 23:00:35.796908 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 23:00:35.796986 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
I20260808 23:00:35.797690 6999 heartbeater.cc:347] Connected to a master server at 127.23.194.62:34357
I20260808 23:00:35.797777 6999 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:35.797948 6999 heartbeater.cc:511] Master 127.23.194.62:34357 requested a full tablet report, sending...
I20260808 23:00:35.798290 6251 ts_manager.cc:194] Registered new tserver with Master: ba35f2602c6546a9ac0ef11bf4f17046 (127.23.194.3:44085)
I20260808 23:00:35.798632 6251 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.194.3:57921
I20260808 23:00:35.805092 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 23:00:35.806878 6321 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:35.806392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49846 (local address 127.23.194.62:34357)
0808 23:00:35.806451 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:35.806453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:35.806576 (+ 123us) server_negotiation.cc:407] Connection header received
0808 23:00:35.806603 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:35.806604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:35.806615 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:35.806695 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:35.806814 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49846: BlockingRecv error: recv got EOF from 127.0.0.1:49846 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 23:00:35.806975 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 6356
I20260808 23:00:35.823768 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 6569
I20260808 23:00:35.832484 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 6792
I20260808 23:00:35.838753 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 6199
2026-08-08T23:00:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (633 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T23:00:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:00:35Z Disabled control of system clock
I20260808 23:00:35.870489 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:37081
--webserver_interface=127.23.194.62
--webserver_port=0
--builtin_ntp_servers=127.23.194.20:34025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:37081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 23:00:35.951081 7020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:35.951294 7020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:35.951315 7020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:35.952391 7020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:00:35.952425 7020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:35.952437 7020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:35.952448 7020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:00:35.952464 7020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:00:35.953799 7020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:34025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:37081
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:37081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:35.954005 7020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:35.954279 7020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:35.959096 7029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:35.959111 7031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:35.959111 7027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:35.959319 7020 server_base.cc:1034] running on GCE node
I20260808 23:00:35.959573 7020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:35.959843 7020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:35.960994 7020 hybrid_clock.cc:648] HybridClock initialized: now 1786230035960979 us; error 28 us; skew 500 ppm
I20260808 23:00:35.962219 7020 webserver.cc:460] Webserver started at http://127.23.194.62:46749/ using document root <none> and password file <none>
I20260808 23:00:35.962436 7020 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:35.962481 7020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:35.962574 7020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:35.963574 7020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "6bc4becbf57541c69f92a3459ce47786"
format_stamp: "Formatted at 2026-08-08 23:00:35 on dist-test-slave-f0tn"
server_key: "216486ec3ce0bf5edcd00a397167f65d"
server_key_iv: "fd04956c3812ad772828a325234f26f6"
server_key_version: "encryptionkey@0"
I20260808 23:00:35.963867 7020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "6bc4becbf57541c69f92a3459ce47786"
format_stamp: "Formatted at 2026-08-08 23:00:35 on dist-test-slave-f0tn"
server_key: "216486ec3ce0bf5edcd00a397167f65d"
server_key_iv: "fd04956c3812ad772828a325234f26f6"
server_key_version: "encryptionkey@0"
I20260808 23:00:35.965045 7020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:00:35.965726 7040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:35.965863 7020 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:35.965940 7020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "6bc4becbf57541c69f92a3459ce47786"
format_stamp: "Formatted at 2026-08-08 23:00:35 on dist-test-slave-f0tn"
server_key: "216486ec3ce0bf5edcd00a397167f65d"
server_key_iv: "fd04956c3812ad772828a325234f26f6"
server_key_version: "encryptionkey@0"
I20260808 23:00:35.966005 7020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:35.969099 7020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:35.969333 7020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:35.969473 7020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:35.973127 7020 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:37081
I20260808 23:00:35.973168 7116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:37081 every 8 connection(s)
I20260808 23:00:35.973485 7020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:00:35.974092 7118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:35.974429 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 7020
I20260808 23:00:35.974512 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:00:35.974700 24328 external_mini_cluster.cc:1428] Setting key 0b4eacc616ca9574f6fa20135b4ddc77
I20260808 23:00:35.977010 7122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:35.975155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58394 (local address 127.23.194.62:37081)
0808 23:00:35.975330 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:35.975337 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:35.975360 (+ 23us) server_negotiation.cc:407] Connection header received
0808 23:00:35.975830 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:35.975836 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:35.975874 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:35.975984 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:35.976597 (+ 613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:35.976600 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:35.976600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:35.976604 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:35.976640 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:35.976653 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:35.976705 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:35.976787 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 23:00:35.976832 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":65,"threads_started":1}
I20260808 23:00:35.977577 7118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786: Bootstrap starting.
I20260808 23:00:35.978163 7118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:35.978412 7118 log.cc:824] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:35.979024 7118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786: No bootstrap required, opened a new log
I20260808 23:00:35.980310 7118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc4becbf57541c69f92a3459ce47786" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 37081 } }
I20260808 23:00:35.980434 7118 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:35.980468 7118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bc4becbf57541c69f92a3459ce47786, State: Initialized, Role: FOLLOWER
I20260808 23:00:35.980578 7118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc4becbf57541c69f92a3459ce47786" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 37081 } }
I20260808 23:00:35.980638 7118 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:00:35.980669 7118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:00:35.980715 7118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:35.981297 7118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc4becbf57541c69f92a3459ce47786" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 37081 } }
I20260808 23:00:35.981431 7118 leader_election.cc:302] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bc4becbf57541c69f92a3459ce47786; no voters:
I20260808 23:00:35.981587 7118 leader_election.cc:288] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:00:35.981617 7125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:35.981768 7125 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786 [term 1 LEADER]: Becoming Leader. State: Replica: 6bc4becbf57541c69f92a3459ce47786, State: Running, Role: LEADER
I20260808 23:00:35.981849 7118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:00:35.981849 7125 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc4becbf57541c69f92a3459ce47786" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 37081 } }
I20260808 23:00:35.982192 7125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bc4becbf57541c69f92a3459ce47786. Latest consensus state: current_term: 1 leader_uuid: "6bc4becbf57541c69f92a3459ce47786" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc4becbf57541c69f92a3459ce47786" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 37081 } } }
I20260808 23:00:35.982265 7125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:35.982223 7129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bc4becbf57541c69f92a3459ce47786" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc4becbf57541c69f92a3459ce47786" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 37081 } } }
I20260808 23:00:35.982465 7129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:35.982488 7134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:00:35.983129 7134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:00:35.984468 7134 catalog_manager.cc:1347] Generated new cluster ID: a021c55cea9b421d9b63da7604efdd9b
I20260808 23:00:35.984512 7134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:00:36.007558 7134 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:00:36.007732 7134 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:00:36.015820 7134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bc4becbf57541c69f92a3459ce47786: Generated new TSK 0
I20260808 23:00:36.015991 7134 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:00:36.020391 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:37081
--builtin_ntp_servers=127.23.194.20:34025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 23:00:36.108145 7154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:36.108348 7154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:36.108369 7154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:36.109567 7154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:36.109609 7154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:36.109635 7154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:00:36.110999 7154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:34025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:37081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:36.111233 7154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:36.111512 7154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:36.116453 7171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:36.116465 7169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:36.116465 7168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:36.116681 7154 server_base.cc:1034] running on GCE node
I20260808 23:00:36.116883 7154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:36.117132 7154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:36.118315 7154 hybrid_clock.cc:648] HybridClock initialized: now 1786230036118302 us; error 31 us; skew 500 ppm
I20260808 23:00:36.128070 7154 webserver.cc:460] Webserver started at http://127.23.194.1:43335/ using document root <none> and password file <none>
I20260808 23:00:36.128319 7154 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:36.128376 7154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:36.128515 7154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:36.129657 7154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "b7ff8e9238334685b32e458bf17c65da"
format_stamp: "Formatted at 2026-08-08 23:00:36 on dist-test-slave-f0tn"
server_key: "3a7451e9f9a8bcf0fff3c9762901aae5"
server_key_iv: "8972561752213fad72c758125ae7c06e"
server_key_version: "encryptionkey@0"
I20260808 23:00:36.130008 7154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "b7ff8e9238334685b32e458bf17c65da"
format_stamp: "Formatted at 2026-08-08 23:00:36 on dist-test-slave-f0tn"
server_key: "3a7451e9f9a8bcf0fff3c9762901aae5"
server_key_iv: "8972561752213fad72c758125ae7c06e"
server_key_version: "encryptionkey@0"
I20260808 23:00:36.131301 7154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:00:36.132057 7178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:36.132254 7154 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:00:36.132354 7154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "b7ff8e9238334685b32e458bf17c65da"
format_stamp: "Formatted at 2026-08-08 23:00:36 on dist-test-slave-f0tn"
server_key: "3a7451e9f9a8bcf0fff3c9762901aae5"
server_key_iv: "8972561752213fad72c758125ae7c06e"
server_key_version: "encryptionkey@0"
I20260808 23:00:36.132418 7154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:36.135831 7154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:36.136096 7154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:36.136214 7154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:36.136405 7154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:36.136708 7154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:36.136742 7154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:36.136770 7154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:36.136793 7154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:36.142513 7154 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:44653
I20260808 23:00:36.142551 7355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:44653 every 8 connection(s)
I20260808 23:00:36.142889 7154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:00:36.145871 7122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:36.143667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:53853 (local address 127.23.194.62:37081)
0808 23:00:36.143748 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:36.143750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:36.143930 (+ 180us) server_negotiation.cc:407] Connection header received
0808 23:00:36.144434 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:36.144435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:36.144441 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:36.144499 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:36.145658 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:36.145660 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:36.145660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:36.145665 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:36.145682 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:36.145686 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:36.145747 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:36.145803 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 23:00:36.145815 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 23:00:36.146039 7359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:36.143624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:37081 (local address 127.23.194.1:53853)
0808 23:00:36.143809 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:36.143848 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:36.144316 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:36.144560 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:36.144563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:36.144572 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:36.145474 (+ 902us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:00:36.145474 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:36.145523 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:00:36.145554 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:36.145748 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:36.145750 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:36.145753 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:00:36.145803 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 23:00:36.145853 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260808 23:00:36.146553 7357 heartbeater.cc:347] Connected to a master server at 127.23.194.62:37081
I20260808 23:00:36.146601 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 7154
I20260808 23:00:36.146660 7357 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:36.146675 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:00:36.146814 7357 heartbeater.cc:511] Master 127.23.194.62:37081 requested a full tablet report, sending...
I20260808 23:00:36.146865 24328 external_mini_cluster.cc:1428] Setting key 105e7bc3d38296dad5d9e35c032b80cf
I20260808 23:00:36.147378 7067 ts_manager.cc:194] Registered new tserver with Master: b7ff8e9238334685b32e458bf17c65da (127.23.194.1:44653)
I20260808 23:00:36.147976 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:37081
--builtin_ntp_servers=127.23.194.20:34025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 23:00:36.229404 7360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:36.229616 7360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:36.229640 7360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:36.230772 7360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:36.230813 7360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:36.230836 7360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:00:36.232151 7360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:34025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:37081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:36.232371 7360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:36.232642 7360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:36.237604 7366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:36.237612 7369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:36.237628 7360 server_base.cc:1034] running on GCE node
W20260808 23:00:36.237620 7367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:36.238108 7360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:36.238349 7360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:36.239491 7360 hybrid_clock.cc:648] HybridClock initialized: now 1786230036239473 us; error 34 us; skew 500 ppm
I20260808 23:00:36.240823 7360 webserver.cc:460] Webserver started at http://127.23.194.2:43201/ using document root <none> and password file <none>
I20260808 23:00:36.241070 7360 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:36.241117 7360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:36.241247 7360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:36.242354 7360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "1a522fa5ad954e7a9644191cc8d12cb4"
format_stamp: "Formatted at 2026-08-08 23:00:36 on dist-test-slave-f0tn"
server_key: "f7697b8078bf64d45b7c4caa7a79155c"
server_key_iv: "c413c2a25cdd66489336513408263df6"
server_key_version: "encryptionkey@0"
I20260808 23:00:36.242689 7360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "1a522fa5ad954e7a9644191cc8d12cb4"
format_stamp: "Formatted at 2026-08-08 23:00:36 on dist-test-slave-f0tn"
server_key: "f7697b8078bf64d45b7c4caa7a79155c"
server_key_iv: "c413c2a25cdd66489336513408263df6"
server_key_version: "encryptionkey@0"
I20260808 23:00:36.243861 7360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:00:36.244511 7379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:36.244647 7360 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:36.244746 7360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "1a522fa5ad954e7a9644191cc8d12cb4"
format_stamp: "Formatted at 2026-08-08 23:00:36 on dist-test-slave-f0tn"
server_key: "f7697b8078bf64d45b7c4caa7a79155c"
server_key_iv: "c413c2a25cdd66489336513408263df6"
server_key_version: "encryptionkey@0"
I20260808 23:00:36.244813 7360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:36.248075 7360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:36.248308 7360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:36.248415 7360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:36.248595 7360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:36.248879 7360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:36.248911 7360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:36.248939 7360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:36.248963 7360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:36.254402 7360 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:43871
I20260808 23:00:36.254438 7557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:43871 every 8 connection(s)
I20260808 23:00:36.254735 7360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:00:36.257360 7122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:36.255396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:54847 (local address 127.23.194.62:37081)
0808 23:00:36.255448 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:36.255450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:36.255666 (+ 216us) server_negotiation.cc:407] Connection header received
0808 23:00:36.256097 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:36.256098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:36.256102 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:36.256173 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:36.257181 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:36.257182 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:36.257183 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:36.257186 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:36.257200 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:36.257204 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:36.257245 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:36.257295 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 23:00:36.257306 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 23:00:36.257579 7561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:36.255373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:37081 (local address 127.23.194.2:54847)
0808 23:00:36.255548 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:36.255586 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:36.256023 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:36.256224 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:36.256227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:36.256234 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:36.257048 (+ 814us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:00:36.257048 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:36.257074 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:00:36.257101 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:36.257244 (+ 143us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:36.257246 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:36.257248 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 23:00:36.257297 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 23:00:36.257343 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260808 23:00:36.258024 7559 heartbeater.cc:347] Connected to a master server at 127.23.194.62:37081
I20260808 23:00:36.258116 7559 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:36.258258 7559 heartbeater.cc:511] Master 127.23.194.62:37081 requested a full tablet report, sending...
I20260808 23:00:36.258581 7067 ts_manager.cc:194] Registered new tserver with Master: 1a522fa5ad954e7a9644191cc8d12cb4 (127.23.194.2:43871)
I20260808 23:00:36.263322 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 7360
I20260808 23:00:36.263408 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:00:36.263579 24328 external_mini_cluster.cc:1428] Setting key dd4351aa52954efe7156668050533f76
I20260808 23:00:36.264407 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:37081
--builtin_ntp_servers=127.23.194.20:34025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 23:00:36.346020 7565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:36.346222 7565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:36.346244 7565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:36.347424 7565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:36.347463 7565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:36.347491 7565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:00:36.348886 7565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:34025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:37081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:36.349117 7565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:36.349432 7565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:36.354220 7572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:36.354177 7576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:36.354199 7581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:36.354372 7565 server_base.cc:1034] running on GCE node
I20260808 23:00:36.354588 7565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:36.354823 7565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:36.355965 7565 hybrid_clock.cc:648] HybridClock initialized: now 1786230036355954 us; error 28 us; skew 500 ppm
I20260808 23:00:36.357175 7565 webserver.cc:460] Webserver started at http://127.23.194.3:39013/ using document root <none> and password file <none>
I20260808 23:00:36.357436 7565 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:36.357498 7565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:36.357628 7565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:36.358645 7565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "ba1186c0454b4579979955aa0f14fb25"
format_stamp: "Formatted at 2026-08-08 23:00:36 on dist-test-slave-f0tn"
server_key: "a55dd90837fd2da9442acba94d60828c"
server_key_iv: "73fca139f3e5de1a0ed8f01a35eb705e"
server_key_version: "encryptionkey@0"
I20260808 23:00:36.358975 7565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "ba1186c0454b4579979955aa0f14fb25"
format_stamp: "Formatted at 2026-08-08 23:00:36 on dist-test-slave-f0tn"
server_key: "a55dd90837fd2da9442acba94d60828c"
server_key_iv: "73fca139f3e5de1a0ed8f01a35eb705e"
server_key_version: "encryptionkey@0"
I20260808 23:00:36.360158 7565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 23:00:36.360781 7593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:36.360930 7565 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:00:36.361011 7565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "ba1186c0454b4579979955aa0f14fb25"
format_stamp: "Formatted at 2026-08-08 23:00:36 on dist-test-slave-f0tn"
server_key: "a55dd90837fd2da9442acba94d60828c"
server_key_iv: "73fca139f3e5de1a0ed8f01a35eb705e"
server_key_version: "encryptionkey@0"
I20260808 23:00:36.361076 7565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:36.364283 7565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:36.364506 7565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:36.364616 7565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:36.364799 7565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:36.365070 7565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:36.365103 7565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:36.365126 7565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:36.365144 7565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:36.370529 7565 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:33277
I20260808 23:00:36.370584 7767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:33277 every 8 connection(s)
I20260808 23:00:36.370903 7565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 23:00:36.373672 7122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:36.371518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:48021 (local address 127.23.194.62:37081)
0808 23:00:36.371560 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:36.371561 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:36.371776 (+ 215us) server_negotiation.cc:407] Connection header received
0808 23:00:36.372277 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:36.372278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:36.372286 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:36.372333 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:36.373424 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:36.373426 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:36.373426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:00:36.373431 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:36.373447 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:00:36.373452 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:36.373502 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:36.373578 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 23:00:36.373592 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 23:00:36.373843 7770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:36.371504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:37081 (local address 127.23.194.3:48021)
0808 23:00:36.371668 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:36.371697 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:36.372163 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:36.372410 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:36.372415 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:36.372424 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:36.373269 (+ 845us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:00:36.373270 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:36.373296 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:00:36.373322 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:36.373505 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:36.373510 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:36.373513 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:00:36.373587 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 23:00:36.373646 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260808 23:00:36.374281 7768 heartbeater.cc:347] Connected to a master server at 127.23.194.62:37081
I20260808 23:00:36.374361 7768 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:36.374500 7768 heartbeater.cc:511] Master 127.23.194.62:37081 requested a full tablet report, sending...
I20260808 23:00:36.374840 7067 ts_manager.cc:194] Registered new tserver with Master: ba1186c0454b4579979955aa0f14fb25 (127.23.194.3:33277)
I20260808 23:00:36.379925 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 7565
I20260808 23:00:36.380003 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:00:36.380203 24328 external_mini_cluster.cc:1428] Setting key 8f77f3221dd707836e00e183674aa8a6
I20260808 23:00:36.380975 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 23:00:36.382829 7122 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:00:36.382348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58398 (local address 127.23.194.62:37081)
0808 23:00:36.382395 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:36.382397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:36.382495 (+ 98us) server_negotiation.cc:407] Connection header received
0808 23:00:36.382579 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:36.382580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:36.382585 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:36.382626 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:36.382763 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58398: BlockingRecv error: recv got EOF from 127.0.0.1:58398 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 23:00:36.382846 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 7154
I20260808 23:00:36.393695 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 7360
I20260808 23:00:36.402179 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 7565
I20260808 23:00:36.408238 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 7020
2026-08-08T23:00:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (565 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:00:36 dist-test-slave-f0tn krb5kdc[7785](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:00:36 dist-test-slave-f0tn krb5kdc[7785](info): set up 2 sockets
Aug 08 23:00:36 dist-test-slave-f0tn krb5kdc[7785](info): commencing operation
krb5kdc: starting...
W20260808 23:00:37.544965 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:00:37 dist-test-slave-f0tn krb5kdc[7785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:00:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:00:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:00:37.659186 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:46535
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:40071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:46535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:37.738317 7806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:37.738510 7806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:37.738531 7806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:37.739596 7806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:00:37.739629 7806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:37.739647 7806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:37.739660 7806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:00:37.739670 7806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:00:37.741060 7806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:40071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:46535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:46535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:37.741240 7806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:37.741425 7806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:37.745991 7816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:37.745977 7820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:37.746049 7806 server_base.cc:1034] running on GCE node
W20260808 23:00:37.745966 7817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:37.746423 7806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:37.746685 7806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:37.747821 7806 hybrid_clock.cc:648] HybridClock initialized: now 1786230037747807 us; error 27 us; skew 500 ppm
Aug 08 23:00:37 dist-test-slave-f0tn krb5kdc[7785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230037, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:37.750075 7806 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:00:37.750484 7806 webserver.cc:460] Webserver started at http://127.23.194.62:44081/ using document root <none> and password file <none>
I20260808 23:00:37.750679 7806 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:37.750726 7806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:37.750846 7806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:37.751828 7806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "4f634296c44745c58484988cde874f43"
format_stamp: "Formatted at 2026-08-08 23:00:37 on dist-test-slave-f0tn"
server_key: "e75220e28f2999cef14565778301fa42"
server_key_iv: "11e578e01bfba2bd3b5567732383ab8f"
server_key_version: "encryptionkey@0"
I20260808 23:00:37.752143 7806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "4f634296c44745c58484988cde874f43"
format_stamp: "Formatted at 2026-08-08 23:00:37 on dist-test-slave-f0tn"
server_key: "e75220e28f2999cef14565778301fa42"
server_key_iv: "11e578e01bfba2bd3b5567732383ab8f"
server_key_version: "encryptionkey@0"
I20260808 23:00:37.753476 7806 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 23:00:37.754194 7831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:37.754357 7806 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:37.754439 7806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "4f634296c44745c58484988cde874f43"
format_stamp: "Formatted at 2026-08-08 23:00:37 on dist-test-slave-f0tn"
server_key: "e75220e28f2999cef14565778301fa42"
server_key_iv: "11e578e01bfba2bd3b5567732383ab8f"
server_key_version: "encryptionkey@0"
I20260808 23:00:37.754505 7806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:37.763211 7806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:37.764801 7806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:37.764923 7806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:37.768699 7806 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:46535
I20260808 23:00:37.768738 7916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:46535 every 8 connection(s)
I20260808 23:00:37.769032 7806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:00:37.769966 7919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:37.772089 7919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43: Bootstrap starting.
I20260808 23:00:37.772610 7919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:37.772840 7919 log.cc:824] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:37.773324 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 7806
I20260808 23:00:37.773438 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:00:37.773494 7919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43: No bootstrap required, opened a new log
I20260808 23:00:37.773627 24328 external_mini_cluster.cc:1428] Setting key cd780ac8a503b3e4db6f4f5da92bd068
I20260808 23:00:37.775569 7919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f634296c44745c58484988cde874f43" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46535 } }
I20260808 23:00:37.775707 7919 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:37.775733 7919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f634296c44745c58484988cde874f43, State: Initialized, Role: FOLLOWER
I20260808 23:00:37.775852 7919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f634296c44745c58484988cde874f43" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46535 } }
I20260808 23:00:37.775924 7919 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:00:37.775949 7919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:00:37.775993 7919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:37.776510 7919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f634296c44745c58484988cde874f43" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46535 } }
I20260808 23:00:37.776594 7919 leader_election.cc:302] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f634296c44745c58484988cde874f43; no voters:
I20260808 23:00:37.776748 7919 leader_election.cc:288] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:00:37.776782 7926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:37.776922 7926 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43 [term 1 LEADER]: Becoming Leader. State: Replica: 4f634296c44745c58484988cde874f43, State: Running, Role: LEADER
I20260808 23:00:37.776997 7919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:00:37.777065 7926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f634296c44745c58484988cde874f43" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46535 } }
I20260808 23:00:37.777411 7926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f634296c44745c58484988cde874f43. Latest consensus state: current_term: 1 leader_uuid: "4f634296c44745c58484988cde874f43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f634296c44745c58484988cde874f43" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46535 } } }
I20260808 23:00:37.777495 7928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f634296c44745c58484988cde874f43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f634296c44745c58484988cde874f43" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46535 } } }
I20260808 23:00:37.777556 7926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:37.777575 7928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43 [sys.catalog]: This master's current role is: LEADER
I20260808 23:00:37.777787 7937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:00:37.778326 7937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:00:37.779862 7937 catalog_manager.cc:1347] Generated new cluster ID: a5bd77f6bf604f9ab8882386968655a5
I20260808 23:00:37.779915 7937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 23:00:37 dist-test-slave-f0tn krb5kdc[7785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:37.783989 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:37.774129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48398 (local address 127.23.194.62:46535)
0808 23:00:37.774311 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:37.774319 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:37.774359 (+ 40us) server_negotiation.cc:407] Connection header received
0808 23:00:37.774934 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:37.774942 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:37.774984 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:37.775105 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:37.775701 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:37.776999 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:37.778088 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:37.778686 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:37.778781 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:37.781085 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:37.781088 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:37.781089 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:37.781100 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:37.782657 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:37.782964 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:37.782966 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:37.782967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:37.783056 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:37.783235 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:37.783236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:37.783236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:37.783452 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:37.783593 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:37.783692 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 23:00:37.783817 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260808 23:00:37.807132 7937 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:00:37.807606 7937 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:00:37.810931 7937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43: Generated new TSK 0
I20260808 23:00:37.811100 7937 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
I20260808 23:00:37.844276 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:46535
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:40071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:37.937516 7960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:37.937722 7960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:37.937745 7960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:37.939097 7960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:37.939155 7960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:37.939181 7960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:00:37.940702 7960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:40071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:46535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:37.940927 7960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:37.941143 7960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:37.945930 7969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:37.945930 7974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:37.945919 7970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:37.945967 7960 server_base.cc:1034] running on GCE node
I20260808 23:00:37.946311 7960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:37.946544 7960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:37.947682 7960 hybrid_clock.cc:648] HybridClock initialized: now 1786230037947669 us; error 32 us; skew 500 ppm
Aug 08 23:00:37 dist-test-slave-f0tn krb5kdc[7785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230037, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:37.950083 7960 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 23:00:37.950498 7960 webserver.cc:460] Webserver started at http://127.23.194.1:43421/ using document root <none> and password file <none>
I20260808 23:00:37.950693 7960 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:37.950740 7960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:37.950861 7960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:37.951846 7960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "5d0811d4f3714cffbb6b10148e7e2f6c"
format_stamp: "Formatted at 2026-08-08 23:00:37 on dist-test-slave-f0tn"
server_key: "96ad40f6eabbc907d03ce368d5fbd0af"
server_key_iv: "dbcb656055d6fcf9aa92d29b9d117e11"
server_key_version: "encryptionkey@0"
I20260808 23:00:37.952167 7960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "5d0811d4f3714cffbb6b10148e7e2f6c"
format_stamp: "Formatted at 2026-08-08 23:00:37 on dist-test-slave-f0tn"
server_key: "96ad40f6eabbc907d03ce368d5fbd0af"
server_key_iv: "dbcb656055d6fcf9aa92d29b9d117e11"
server_key_version: "encryptionkey@0"
I20260808 23:00:37.953339 7960 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 23:00:37.954012 7981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:37.954173 7960 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:37.954257 7960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "5d0811d4f3714cffbb6b10148e7e2f6c"
format_stamp: "Formatted at 2026-08-08 23:00:37 on dist-test-slave-f0tn"
server_key: "96ad40f6eabbc907d03ce368d5fbd0af"
server_key_iv: "dbcb656055d6fcf9aa92d29b9d117e11"
server_key_version: "encryptionkey@0"
I20260808 23:00:37.954316 7960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:37.963464 7960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:37.965224 7960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:37.965348 7960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:37.965562 7960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:37.965844 7960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:37.965878 7960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:37.965904 7960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:37.965921 7960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:37.971370 7960 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:36899
I20260808 23:00:37.971429 8160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:36899 every 8 connection(s)
I20260808 23:00:37.971699 7960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
Aug 08 23:00:37 dist-test-slave-f0tn krb5kdc[7785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230037, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:37.979717 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 7960
I20260808 23:00:37.979821 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:00:37.980026 24328 external_mini_cluster.cc:1428] Setting key bc876adcc091e32dfa16c942ffd1fa85
I20260808 23:00:37.980203 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:37.972342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:33461 (local address 127.23.194.62:46535)
0808 23:00:37.972392 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:37.972394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:37.972596 (+ 202us) server_negotiation.cc:407] Connection header received
0808 23:00:37.973048 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:37.973049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:37.973057 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:37.973107 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:37.974015 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:37.974615 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:37.975564 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:37.975770 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:37.975820 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:37.977787 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:37.977789 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:37.977789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:37.977798 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:37.978998 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:37.979350 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:37.979350 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:37.979351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:37.979398 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:37.979588 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:37.979588 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:37.979589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:37.979690 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:37.979736 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:37.980038 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 23:00:37.980129 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 23:00:37.980392 8165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:37.972319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:46535 (local address 127.23.194.1:33461)
0808 23:00:37.972476 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:37.972517 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:37.972975 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:37.973173 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:37.973176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:37.973758 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:37.973942 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:37.973947 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:37.974711 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:37.974712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:37.975498 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:37.975500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:37.975834 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:37.975835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:37.975930 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:37.976323 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:37.976331 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:37.977699 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:37.979068 (+ 1369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:37.979074 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:37.979076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:37.979295 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:37.979440 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:37.979441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:37.979441 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:37.979527 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:37.979742 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:37.979748 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:37.979907 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 23:00:37.980018 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 23:00:37.980180 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
I20260808 23:00:37.980911 8162 heartbeater.cc:347] Connected to a master server at 127.23.194.62:46535
I20260808 23:00:37.981009 8162 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:37.981184 8162 heartbeater.cc:511] Master 127.23.194.62:46535 requested a full tablet report, sending...
I20260808 23:00:37.981705 7857 ts_manager.cc:194] Registered new tserver with Master: 5d0811d4f3714cffbb6b10148e7e2f6c (127.23.194.1:36899)
I20260808 23:00:37.982293 7857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:33461
WARNING: no policy specified for kudu/127.23.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
I20260808 23:00:38.004189 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:46535
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:40071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:38.084784 8174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:38.084986 8174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:38.085008 8174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:38.086180 8174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:38.086220 8174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:38.086247 8174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:00:38.087692 8174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:40071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:46535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:38.087908 8174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:38.088121 8174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:38.092988 8183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:38.092988 8185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:38.093101 8174 server_base.cc:1034] running on GCE node
W20260808 23:00:38.093076 8182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:38.093444 8174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:38.093693 8174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:38.094825 8174 hybrid_clock.cc:648] HybridClock initialized: now 1786230038094807 us; error 33 us; skew 500 ppm
Aug 08 23:00:38 dist-test-slave-f0tn krb5kdc[7785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230038, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:38.097216 8174 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 23:00:38.097678 8174 webserver.cc:460] Webserver started at http://127.23.194.2:44211/ using document root <none> and password file <none>
I20260808 23:00:38.097882 8174 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:38.097930 8174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:38.098052 8174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:38.099062 8174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "83bba5e22d594e8c85f696379fa81e54"
format_stamp: "Formatted at 2026-08-08 23:00:38 on dist-test-slave-f0tn"
server_key: "eb70485bae674b78ef1741e02f21328e"
server_key_iv: "bd9dd8cb7e8077fa5595cd706045fe4c"
server_key_version: "encryptionkey@0"
I20260808 23:00:38.099400 8174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "83bba5e22d594e8c85f696379fa81e54"
format_stamp: "Formatted at 2026-08-08 23:00:38 on dist-test-slave-f0tn"
server_key: "eb70485bae674b78ef1741e02f21328e"
server_key_iv: "bd9dd8cb7e8077fa5595cd706045fe4c"
server_key_version: "encryptionkey@0"
I20260808 23:00:38.100697 8174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:00:38.101366 8195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:38.101539 8174 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:00:38.101626 8174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "83bba5e22d594e8c85f696379fa81e54"
format_stamp: "Formatted at 2026-08-08 23:00:38 on dist-test-slave-f0tn"
server_key: "eb70485bae674b78ef1741e02f21328e"
server_key_iv: "bd9dd8cb7e8077fa5595cd706045fe4c"
server_key_version: "encryptionkey@0"
I20260808 23:00:38.101697 8174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:38.134384 8174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:38.136242 8174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:38.136389 8174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:38.136626 8174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:38.136947 8174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:38.136991 8174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:38.137023 8174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:38.137048 8174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:38.142802 8174 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:36389
I20260808 23:00:38.142882 8368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:36389 every 8 connection(s)
I20260808 23:00:38.143196 8174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
Aug 08 23:00:38 dist-test-slave-f0tn krb5kdc[7785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230038, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:38.149479 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 8174
I20260808 23:00:38.149561 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:00:38.149749 24328 external_mini_cluster.cc:1428] Setting key c15a6271844d6152c53d6bca050b18a4
I20260808 23:00:38.152734 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:38.143948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:47485 (local address 127.23.194.62:46535)
0808 23:00:38.144018 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:38.144020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:38.144213 (+ 193us) server_negotiation.cc:407] Connection header received
0808 23:00:38.144708 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:38.144709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:38.144720 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:38.144787 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:38.145854 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.146383 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.147370 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.147567 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.147632 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:38.149851 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:38.149853 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:38.149853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:38.149862 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:38.151430 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:38.151813 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:38.151814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:38.151814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:38.151879 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:38.152079 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:38.152080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:38.152080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:38.152216 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:38.152283 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:38.152597 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 23:00:38.152667 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 23:00:38.152992 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:38.143911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:46535 (local address 127.23.194.2:47485)
0808 23:00:38.144086 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:38.144130 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:38.144612 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:38.144853 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:38.144857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:38.145498 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:38.145723 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:38.145729 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.146463 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.146463 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:38.147265 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:38.147267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.147651 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.147652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:38.147755 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:38.148263 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:38.148272 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:38.149752 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:38.151506 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:38.151513 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:38.151515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:38.151745 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:38.151935 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:38.151936 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:38.151936 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:38.152033 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:38.152281 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:38.152287 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:38.152462 (+ 175us) client_negotiation.cc:769] Sending connection context
0808 23:00:38.152577 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 23:00:38.152750 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260808 23:00:38.153604 8369 heartbeater.cc:347] Connected to a master server at 127.23.194.62:46535
I20260808 23:00:38.153707 8369 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:38.153884 8369 heartbeater.cc:511] Master 127.23.194.62:46535 requested a full tablet report, sending...
I20260808 23:00:38.154302 7857 ts_manager.cc:194] Registered new tserver with Master: 83bba5e22d594e8c85f696379fa81e54 (127.23.194.2:36389)
I20260808 23:00:38.154692 7857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:47485
WARNING: no policy specified for kudu/127.23.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
I20260808 23:00:38.174322 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:46535
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:40071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:00:38.255366 8377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:00:38.255564 8377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:00:38.255585 8377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:00:38.256671 8377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:00:38.256711 8377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:00:38.256734 8377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:00:38.258222 8377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:40071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:46535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:00:38.258411 8377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:00:38.258582 8377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:00:38.263492 8383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:38.263482 8385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:00:38.263466 8388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:00:38.263666 8377 server_base.cc:1034] running on GCE node
I20260808 23:00:38.263860 8377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:00:38.264087 8377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:00:38.265218 8377 hybrid_clock.cc:648] HybridClock initialized: now 1786230038265203 us; error 32 us; skew 500 ppm
Aug 08 23:00:38 dist-test-slave-f0tn krb5kdc[7785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230038, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:00:38.267735 8377 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 23:00:38.268175 8377 webserver.cc:460] Webserver started at http://127.23.194.3:43533/ using document root <none> and password file <none>
I20260808 23:00:38.268388 8377 fs_manager.cc:359] Metadata directory not provided
I20260808 23:00:38.268432 8377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:00:38.268558 8377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:00:38.269676 8377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "be201d2ecc2a4d0380cba6dcf679d3e3"
format_stamp: "Formatted at 2026-08-08 23:00:38 on dist-test-slave-f0tn"
server_key: "65d01af142caf1318e92dcb75c423e91"
server_key_iv: "1cdf92761e3255c70d0d5a579d775b67"
server_key_version: "encryptionkey@0"
I20260808 23:00:38.270005 8377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "be201d2ecc2a4d0380cba6dcf679d3e3"
format_stamp: "Formatted at 2026-08-08 23:00:38 on dist-test-slave-f0tn"
server_key: "65d01af142caf1318e92dcb75c423e91"
server_key_iv: "1cdf92761e3255c70d0d5a579d775b67"
server_key_version: "encryptionkey@0"
I20260808 23:00:38.271306 8377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:00:38.272014 8398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:38.272171 8377 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:38.272248 8377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "be201d2ecc2a4d0380cba6dcf679d3e3"
format_stamp: "Formatted at 2026-08-08 23:00:38 on dist-test-slave-f0tn"
server_key: "65d01af142caf1318e92dcb75c423e91"
server_key_iv: "1cdf92761e3255c70d0d5a579d775b67"
server_key_version: "encryptionkey@0"
I20260808 23:00:38.272313 8377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:00:38.282150 8377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:00:38.284430 8377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:00:38.284557 8377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:00:38.284770 8377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:00:38.285074 8377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:00:38.285115 8377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:38.285146 8377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:00:38.285171 8377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:00:38.290810 8377 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:41007
I20260808 23:00:38.290848 8591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:41007 every 8 connection(s)
I20260808 23:00:38.291152 8377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
Aug 08 23:00:38 dist-test-slave-f0tn krb5kdc[7785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230038, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:00:38.299597 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:38.291833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:55563 (local address 127.23.194.62:46535)
0808 23:00:38.291879 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:38.291880 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:38.292103 (+ 223us) server_negotiation.cc:407] Connection header received
0808 23:00:38.292623 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:38.292625 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:38.292637 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:38.292688 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:38.293675 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.294177 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.295132 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.295324 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.295379 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:38.297358 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:38.297359 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:38.297359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:38.297366 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:38.298565 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:38.298918 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:38.298918 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:38.298918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:38.298964 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:38.299133 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:38.299133 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:38.299134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:38.299239 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:38.299284 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:38.299476 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 23:00:38.299531 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 23:00:38.299801 8596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:38.291786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:46535 (local address 127.23.194.3:55563)
0808 23:00:38.291978 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:38.292015 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:38.292505 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:38.292758 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:38.292763 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:38.293336 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:00:38.293590 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:38.293595 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.294260 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.294261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:38.295051 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:38.295053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.295391 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.295392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:38.295509 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:38.295966 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:00:38.295974 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:00:38.297273 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:00:38.298653 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:38.298658 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:38.298660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:38.298865 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:38.299006 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:00:38.299007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:00:38.299007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:00:38.299093 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:00:38.299280 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:00:38.299283 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:00:38.299387 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 23:00:38.299460 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 23:00:38.299571 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":97,"threads_started":1}
I20260808 23:00:38.299902 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 8377
I20260808 23:00:38.299994 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:00:38.300203 24328 external_mini_cluster.cc:1428] Setting key 4ffa30db68e0db1ba4b8f69d766814bb
I20260808 23:00:38.300395 8592 heartbeater.cc:347] Connected to a master server at 127.23.194.62:46535
I20260808 23:00:38.300478 8592 heartbeater.cc:464] Registering TS with master...
I20260808 23:00:38.300668 8592 heartbeater.cc:511] Master 127.23.194.62:46535 requested a full tablet report, sending...
I20260808 23:00:38.301071 7857 ts_manager.cc:194] Registered new tserver with Master: be201d2ecc2a4d0380cba6dcf679d3e3 (127.23.194.3:41007)
I20260808 23:00:38.301457 7857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:55563
I20260808 23:00:38.311803 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:00:38.318292 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:38.313064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48404 (local address 127.23.194.62:46535)
0808 23:00:38.313108 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:38.313110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:38.313193 (+ 83us) server_negotiation.cc:407] Connection header received
0808 23:00:38.313236 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:38.313236 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:38.313245 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:38.313292 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:00:38.313821 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.314304 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.314961 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.315129 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.315181 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:38.316200 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:00:38.316201 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:00:38.316202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:00:38.316208 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:00:38.317359 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:38.317685 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:38.317686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:38.317686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:38.317733 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:00:38.317855 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:00:38.317855 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:00:38.317855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:00:38.317991 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:00:38.318062 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:38.318150 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 23:00:38.318226 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 23:00:38.319732 7857 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:00:38.321305 7857 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:00:38.327971 8609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:38.323599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:36389 (local address 127.0.0.1:41494)
0808 23:00:38.323934 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:38.323976 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:38.324043 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:38.324271 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:38.324275 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:38.324278 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:38.324352 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:38.324354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.325728 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.325730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:38.326776 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:38.326779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.327659 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.327661 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:38.327744 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:38.327759 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 23:00:38.327825 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 23:00:38.327882 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":46,"threads_started":1}
I20260808 23:00:38.328142 8613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:38.323609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:41007 (local address 127.0.0.1:60736)
0808 23:00:38.323933 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:38.323977 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:38.324043 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:38.324408 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:38.324408 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:38.324410 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:38.324451 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:38.324452 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.325731 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.325763 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:38.327059 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:38.327062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.327945 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.327946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:38.327999 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:38.328016 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 23:00:38.328045 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 23:00:38.328082 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":78,"threads_started":1}
I20260808 23:00:38.328307 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:38.323788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41494 (local address 127.23.194.2:36389)
0808 23:00:38.324070 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:38.324076 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:38.324088 (+ 12us) server_negotiation.cc:407] Connection header received
0808 23:00:38.324131 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:38.324134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:38.324150 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:38.324233 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:38.324412 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.325601 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.326855 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.327479 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.327561 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:38.327622 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:38.328183 (+ 561us) server_negotiation.cc:299] Negotiation successful
0808 23:00:38.328231 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":91,"threads_started":1}
I20260808 23:00:38.328524 8612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:38.323717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60736 (local address 127.23.194.3:41007)
0808 23:00:38.324184 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:38.324189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:38.324197 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:00:38.324237 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:38.324239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:38.324255 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:38.324324 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:38.324520 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.325645 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.327151 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.327859 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.328309 (+ 450us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:38.328383 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:38.328408 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 23:00:38.328450 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":109,"threads_started":1}
I20260808 23:00:38.329054 8611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:38.323814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:36899 (local address 127.0.0.1:60966)
0808 23:00:38.324543 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:38.324551 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:38.324592 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:38.324796 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:38.324797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:38.324799 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:38.324841 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:38.324843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.327200 (+ 2357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.327200 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:38.327891 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:38.327893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.328809 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.328810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:38.328897 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:38.328901 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:00:38.328944 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 23:00:38.328984 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"thread_start_us":63,"threads_started":1}
I20260808 23:00:38.329108 8615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:38.323934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60966 (local address 127.23.194.1:36899)
0808 23:00:38.324116 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:38.324120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:38.324665 (+ 545us) server_negotiation.cc:407] Connection header received
0808 23:00:38.324706 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:38.324708 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:38.324721 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:38.324783 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:38.326004 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.327112 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.327973 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.328727 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.328807 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:38.328861 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:38.328984 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 23:00:38.329027 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":73,"threads_started":1}
I20260808 23:00:38.329084 8274 tablet_service.cc:1467] Processing CreateTablet for tablet 25c6e6c23f9b463ba1abbb12554cb8e0 (DEFAULT_TABLE table=test-table [id=1e4dfcc9ac1c4d7a9750974da9cb0568]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:38.329501 8274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25c6e6c23f9b463ba1abbb12554cb8e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:38.329825 8058 tablet_service.cc:1467] Processing CreateTablet for tablet 25c6e6c23f9b463ba1abbb12554cb8e0 (DEFAULT_TABLE table=test-table [id=1e4dfcc9ac1c4d7a9750974da9cb0568]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:38.330034 8487 tablet_service.cc:1467] Processing CreateTablet for tablet 25c6e6c23f9b463ba1abbb12554cb8e0 (DEFAULT_TABLE table=test-table [id=1e4dfcc9ac1c4d7a9750974da9cb0568]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:00:38.330161 8058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25c6e6c23f9b463ba1abbb12554cb8e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:38.330372 8487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25c6e6c23f9b463ba1abbb12554cb8e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:00:38.332967 8617 tablet_bootstrap.cc:492] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54: Bootstrap starting.
I20260808 23:00:38.333174 8616 tablet_bootstrap.cc:492] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 5d0811d4f3714cffbb6b10148e7e2f6c: Bootstrap starting.
I20260808 23:00:38.333582 8619 tablet_bootstrap.cc:492] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3: Bootstrap starting.
I20260808 23:00:38.333669 8617 tablet_bootstrap.cc:654] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:38.333854 8616 tablet_bootstrap.cc:654] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 5d0811d4f3714cffbb6b10148e7e2f6c: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:38.334271 8616 log.cc:824] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 5d0811d4f3714cffbb6b10148e7e2f6c: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:38.334256 8617 log.cc:824] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:38.334482 8619 tablet_bootstrap.cc:654] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3: Neither blocks nor log segments found. Creating new log.
I20260808 23:00:38.334969 8616 tablet_bootstrap.cc:492] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 5d0811d4f3714cffbb6b10148e7e2f6c: No bootstrap required, opened a new log
I20260808 23:00:38.335047 8616 ts_tablet_manager.cc:1397] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 5d0811d4f3714cffbb6b10148e7e2f6c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:00:38.335038 8619 log.cc:824] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3: Log is configured to *not* fsync() on all Append() calls
I20260808 23:00:38.335160 8617 tablet_bootstrap.cc:492] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54: No bootstrap required, opened a new log
I20260808 23:00:38.335269 8617 ts_tablet_manager.cc:1397] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 23:00:38.335759 8619 tablet_bootstrap.cc:492] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3: No bootstrap required, opened a new log
I20260808 23:00:38.335832 8619 ts_tablet_manager.cc:1397] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:00:38.336699 8617 raft_consensus.cc:348] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83bba5e22d594e8c85f696379fa81e54" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36389 } } peers { permanent_uuid: "be201d2ecc2a4d0380cba6dcf679d3e3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 41007 } } peers { permanent_uuid: "5d0811d4f3714cffbb6b10148e7e2f6c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36899 } }
I20260808 23:00:38.336839 8617 raft_consensus.cc:374] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:38.336872 8617 raft_consensus.cc:729] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83bba5e22d594e8c85f696379fa81e54, State: Initialized, Role: FOLLOWER
I20260808 23:00:38.336997 8617 consensus_queue.cc:260] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83bba5e22d594e8c85f696379fa81e54" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36389 } } peers { permanent_uuid: "be201d2ecc2a4d0380cba6dcf679d3e3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 41007 } } peers { permanent_uuid: "5d0811d4f3714cffbb6b10148e7e2f6c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36899 } }
I20260808 23:00:38.336949 8616 raft_consensus.cc:348] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 5d0811d4f3714cffbb6b10148e7e2f6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83bba5e22d594e8c85f696379fa81e54" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36389 } } peers { permanent_uuid: "be201d2ecc2a4d0380cba6dcf679d3e3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 41007 } } peers { permanent_uuid: "5d0811d4f3714cffbb6b10148e7e2f6c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36899 } }
I20260808 23:00:38.337098 8616 raft_consensus.cc:374] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 5d0811d4f3714cffbb6b10148e7e2f6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:38.337141 8616 raft_consensus.cc:729] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 5d0811d4f3714cffbb6b10148e7e2f6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d0811d4f3714cffbb6b10148e7e2f6c, State: Initialized, Role: FOLLOWER
I20260808 23:00:38.337245 8616 consensus_queue.cc:260] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 5d0811d4f3714cffbb6b10148e7e2f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83bba5e22d594e8c85f696379fa81e54" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36389 } } peers { permanent_uuid: "be201d2ecc2a4d0380cba6dcf679d3e3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 41007 } } peers { permanent_uuid: "5d0811d4f3714cffbb6b10148e7e2f6c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36899 } }
I20260808 23:00:38.337293 8617 ts_tablet_manager.cc:1428] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 23:00:38.337481 8369 heartbeater.cc:503] Master 127.23.194.62:46535 was elected leader, sending a full tablet report...
I20260808 23:00:38.337559 8616 ts_tablet_manager.cc:1428] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 5d0811d4f3714cffbb6b10148e7e2f6c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 23:00:38.337697 8162 heartbeater.cc:503] Master 127.23.194.62:46535 was elected leader, sending a full tablet report...
I20260808 23:00:38.337719 8619 raft_consensus.cc:348] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83bba5e22d594e8c85f696379fa81e54" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36389 } } peers { permanent_uuid: "be201d2ecc2a4d0380cba6dcf679d3e3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 41007 } } peers { permanent_uuid: "5d0811d4f3714cffbb6b10148e7e2f6c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36899 } }
I20260808 23:00:38.337860 8619 raft_consensus.cc:374] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:00:38.337885 8619 raft_consensus.cc:729] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be201d2ecc2a4d0380cba6dcf679d3e3, State: Initialized, Role: FOLLOWER
I20260808 23:00:38.337994 8619 consensus_queue.cc:260] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83bba5e22d594e8c85f696379fa81e54" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36389 } } peers { permanent_uuid: "be201d2ecc2a4d0380cba6dcf679d3e3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 41007 } } peers { permanent_uuid: "5d0811d4f3714cffbb6b10148e7e2f6c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36899 } }
I20260808 23:00:38.338267 8619 ts_tablet_manager.cc:1428] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 23:00:38.338280 8592 heartbeater.cc:503] Master 127.23.194.62:46535 was elected leader, sending a full tablet report...
I20260808 23:00:38.380519 8626 raft_consensus.cc:482] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:00:38.380673 8626 raft_consensus.cc:504] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83bba5e22d594e8c85f696379fa81e54" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36389 } } peers { permanent_uuid: "be201d2ecc2a4d0380cba6dcf679d3e3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 41007 } } peers { permanent_uuid: "5d0811d4f3714cffbb6b10148e7e2f6c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36899 } }
I20260808 23:00:38.381055 8626 leader_election.cc:288] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83bba5e22d594e8c85f696379fa81e54 (127.23.194.2:36389), 5d0811d4f3714cffbb6b10148e7e2f6c (127.23.194.1:36899)
I20260808 23:00:38.384761 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:38.381298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:58785 (local address 127.23.194.2:36389)
0808 23:00:38.381401 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:38.381403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:38.381410 (+ 7us) server_negotiation.cc:407] Connection header received
0808 23:00:38.381435 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:38.381436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:38.381449 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:38.381504 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:38.381662 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.382414 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.383840 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.384420 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.384496 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:38.384505 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:38.384653 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 23:00:38.384689 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 23:00:38.384761 8615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:38.381408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:33287 (local address 127.23.194.1:36899)
0808 23:00:38.381468 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:38.381470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:38.381573 (+ 103us) server_negotiation.cc:407] Connection header received
0808 23:00:38.381619 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:38.381619 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:38.381630 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:38.381695 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:00:38.381835 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.382500 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.383841 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.384419 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.384500 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:38.384506 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:38.384654 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 23:00:38.384689 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 23:00:38.384780 8596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:38.381202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:36389 (local address 127.23.194.3:58785)
0808 23:00:38.381259 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:38.381272 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:38.381322 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:38.381515 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:38.381517 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:38.381519 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:38.381594 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:38.381596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.382487 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.382487 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:38.383749 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:38.383753 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.384496 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.384498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:38.384585 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:38.384587 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 23:00:38.384640 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 23:00:38.384683 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":58}
I20260808 23:00:38.384789 8627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:38.381297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:36899 (local address 127.23.194.3:33287)
0808 23:00:38.381506 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:38.381515 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:00:38.381570 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:00:38.381713 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:00:38.381714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:00:38.381716 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:00:38.381779 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:38.381780 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.382561 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.382562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:38.383749 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:00:38.383753 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.384499 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.384499 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:00:38.384582 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:38.384586 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:00:38.384640 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 23:00:38.384682 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":27,"thread_start_us":138,"threads_started":1}
I20260808 23:00:38.385124 8306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c6e6c23f9b463ba1abbb12554cb8e0" candidate_uuid: "be201d2ecc2a4d0380cba6dcf679d3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83bba5e22d594e8c85f696379fa81e54" is_pre_election: true
I20260808 23:00:38.385175 8092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c6e6c23f9b463ba1abbb12554cb8e0" candidate_uuid: "be201d2ecc2a4d0380cba6dcf679d3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d0811d4f3714cffbb6b10148e7e2f6c" is_pre_election: true
I20260808 23:00:38.385289 8306 raft_consensus.cc:2454] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be201d2ecc2a4d0380cba6dcf679d3e3 in term 0.
I20260808 23:00:38.385324 8092 raft_consensus.cc:2454] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 5d0811d4f3714cffbb6b10148e7e2f6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be201d2ecc2a4d0380cba6dcf679d3e3 in term 0.
I20260808 23:00:38.385520 8402 leader_election.cc:302] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83bba5e22d594e8c85f696379fa81e54, be201d2ecc2a4d0380cba6dcf679d3e3; no voters:
I20260808 23:00:38.385648 8626 raft_consensus.cc:2781] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:00:38.385710 8626 raft_consensus.cc:482] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:00:38.385733 8626 raft_consensus.cc:3037] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:38.386441 8626 raft_consensus.cc:504] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83bba5e22d594e8c85f696379fa81e54" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36389 } } peers { permanent_uuid: "be201d2ecc2a4d0380cba6dcf679d3e3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 41007 } } peers { permanent_uuid: "5d0811d4f3714cffbb6b10148e7e2f6c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36899 } }
I20260808 23:00:38.386559 8626 leader_election.cc:288] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [CANDIDATE]: Term 1 election: Requested vote from peers 83bba5e22d594e8c85f696379fa81e54 (127.23.194.2:36389), 5d0811d4f3714cffbb6b10148e7e2f6c (127.23.194.1:36899)
I20260808 23:00:38.386767 8306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c6e6c23f9b463ba1abbb12554cb8e0" candidate_uuid: "be201d2ecc2a4d0380cba6dcf679d3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83bba5e22d594e8c85f696379fa81e54"
I20260808 23:00:38.386767 8092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c6e6c23f9b463ba1abbb12554cb8e0" candidate_uuid: "be201d2ecc2a4d0380cba6dcf679d3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d0811d4f3714cffbb6b10148e7e2f6c"
I20260808 23:00:38.386849 8306 raft_consensus.cc:3037] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:38.386849 8092 raft_consensus.cc:3037] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 5d0811d4f3714cffbb6b10148e7e2f6c [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:00:38.387729 8092 raft_consensus.cc:2454] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 5d0811d4f3714cffbb6b10148e7e2f6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be201d2ecc2a4d0380cba6dcf679d3e3 in term 1.
I20260808 23:00:38.387729 8306 raft_consensus.cc:2454] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be201d2ecc2a4d0380cba6dcf679d3e3 in term 1.
I20260808 23:00:38.387878 8402 leader_election.cc:302] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83bba5e22d594e8c85f696379fa81e54, be201d2ecc2a4d0380cba6dcf679d3e3; no voters:
I20260808 23:00:38.387952 8626 raft_consensus.cc:2781] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:00:38.388123 8626 raft_consensus.cc:686] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [term 1 LEADER]: Becoming Leader. State: Replica: be201d2ecc2a4d0380cba6dcf679d3e3, State: Running, Role: LEADER
I20260808 23:00:38.388224 8626 consensus_queue.cc:237] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83bba5e22d594e8c85f696379fa81e54" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36389 } } peers { permanent_uuid: "be201d2ecc2a4d0380cba6dcf679d3e3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 41007 } } peers { permanent_uuid: "5d0811d4f3714cffbb6b10148e7e2f6c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36899 } }
I20260808 23:00:38.389046 7857 catalog_manager.cc:5495] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to be201d2ecc2a4d0380cba6dcf679d3e3 (127.23.194.3). New cstate: current_term: 1 leader_uuid: "be201d2ecc2a4d0380cba6dcf679d3e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83bba5e22d594e8c85f696379fa81e54" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be201d2ecc2a4d0380cba6dcf679d3e3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 41007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d0811d4f3714cffbb6b10148e7e2f6c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36899 } health_report { overall_health: UNKNOWN } } }
W20260808 23:00:38.393719 8370 tablet.cc:2369] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:00:38.399394 8612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:38.397421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60746 (local address 127.23.194.3:41007)
0808 23:00:38.397461 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 23:00:38.397463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:00:38.397470 (+ 7us) server_negotiation.cc:407] Connection header received
0808 23:00:38.397513 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:00:38.397513 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:00:38.397529 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:00:38.397577 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:00:38.397679 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.398115 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.398821 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:00:38.398993 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:00:38.399035 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:00:38.399125 (+ 90us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:00:38.399227 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:00:38.399282 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 23:00:38.399296 (+ 14us) server_negotiation.cc:299] Negotiation successful
0808 23:00:38.399311 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 23:00:38.400988 8092 raft_consensus.cc:1261] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 5d0811d4f3714cffbb6b10148e7e2f6c [term 1 FOLLOWER]: Refusing update from remote peer be201d2ecc2a4d0380cba6dcf679d3e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:00:38.400988 8306 raft_consensus.cc:1261] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54 [term 1 FOLLOWER]: Refusing update from remote peer be201d2ecc2a4d0380cba6dcf679d3e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:00:38.401278 8626 consensus_queue.cc:1035] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83bba5e22d594e8c85f696379fa81e54" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:38.401417 8629 consensus_queue.cc:1035] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d0811d4f3714cffbb6b10148e7e2f6c" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:00:38.403860 8637 mvcc.cc:205] Tried to move back new op lower bound from 7316398237288304640 to 7316398237238796288. Current Snapshot: MvccSnapshot[applied={T|T < 7316398237288304640}]
I20260808 23:00:38.405476 7855 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:00:38.405575 7855 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:00:38.406284 7855 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f634296c44745c58484988cde874f43: Sending DeleteTablet for 3 replicas of tablet 25c6e6c23f9b463ba1abbb12554cb8e0
I20260808 23:00:38.406569 8487 tablet_service.cc:1514] Processing DeleteTablet for tablet 25c6e6c23f9b463ba1abbb12554cb8e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:38 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:60736
I20260808 23:00:38.406611 8058 tablet_service.cc:1514] Processing DeleteTablet for tablet 25c6e6c23f9b463ba1abbb12554cb8e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:38 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:60966
I20260808 23:00:38.406800 8649 tablet_replica.cc:330] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3: stopping tablet replica
I20260808 23:00:38.406875 8274 tablet_service.cc:1514] Processing DeleteTablet for tablet 25c6e6c23f9b463ba1abbb12554cb8e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:00:38 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:41494
I20260808 23:00:38.406905 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 7960
I20260808 23:00:38.416859 8649 raft_consensus.cc:2229] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:00:38.417047 8649 raft_consensus.cc:2258] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:38.422245 8649 ts_tablet_manager.cc:1905] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:38.423808 8653 tablet_replica.cc:330] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54: stopping tablet replica
I20260808 23:00:38.423914 8653 raft_consensus.cc:2229] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:00:38.424002 8653 raft_consensus.cc:2258] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:00:38.424167 8649 ts_tablet_manager.cc:1918] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:00:38.424218 8649 log.cc:1196] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/25c6e6c23f9b463ba1abbb12554cb8e0
I20260808 23:00:38.424431 8649 ts_tablet_manager.cc:1939] T 25c6e6c23f9b463ba1abbb12554cb8e0 P be201d2ecc2a4d0380cba6dcf679d3e3: Deleting consensus metadata
I20260808 23:00:38.424789 7833 catalog_manager.cc:4841] TS be201d2ecc2a4d0380cba6dcf679d3e3 (127.23.194.3:41007): tablet 25c6e6c23f9b463ba1abbb12554cb8e0 (table test-table [id=1e4dfcc9ac1c4d7a9750974da9cb0568]) successfully deleted
I20260808 23:00:38.426309 8653 ts_tablet_manager.cc:1905] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:00:38.428151 8653 ts_tablet_manager.cc:1918] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:00:38.428244 8653 log.cc:1196] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/25c6e6c23f9b463ba1abbb12554cb8e0
I20260808 23:00:38.428463 8653 ts_tablet_manager.cc:1939] T 25c6e6c23f9b463ba1abbb12554cb8e0 P 83bba5e22d594e8c85f696379fa81e54: Deleting consensus metadata
I20260808 23:00:38.430421 7836 catalog_manager.cc:4841] TS 83bba5e22d594e8c85f696379fa81e54 (127.23.194.2:36389): tablet 25c6e6c23f9b463ba1abbb12554cb8e0 (table test-table [id=1e4dfcc9ac1c4d7a9750974da9cb0568]) successfully deleted
W20260808 23:00:38.434293 7836 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.194.1:36899 (error 108)
I20260808 23:00:38.434494 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 8174
I20260808 23:00:38.443274 8611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:00:38.443087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:36899 (local address 127.0.0.1:60976)
0808 23:00:38.443166 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 23:00:38.443231 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.1:36899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 23:00:38.443822 7836 catalog_manager.cc:4584] TS 5d0811d4f3714cffbb6b10148e7e2f6c (127.23.194.1:36899): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 25c6e6c23f9b463ba1abbb12554cb8e0: Network error: Client connection negotiation failed: client connection to 127.23.194.1:36899: connect: Connection refused (error 111)
I20260808 23:00:38.454192 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 8377
I20260808 23:00:38.466017 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 7806
2026-08-08T23:00:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2061 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:00:38 dist-test-slave-f0tn krb5kdc[8658](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:00:38 dist-test-slave-f0tn krb5kdc[8658](info): set up 2 sockets
Aug 08 23:00:38 dist-test-slave-f0tn krb5kdc[8658](info): commencing operation
krb5kdc: starting...
W20260808 23:00:39.596652 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:00:39 dist-test-slave-f0tn krb5kdc[8658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:00:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:00:39Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.23.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.23.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.23.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/rangeradmin_127.23.194.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.23.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/rangeradmin_127.23.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.23.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.23.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.23.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/rangerlookup_127.23.194.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.23.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/rangerlookup_127.23.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.23.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/HTTP_127.23.194.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/HTTP_127.23.194.20@KRBTEST.COM.keytab.
I20260808 23:00:39.756840 24328 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taski6UDxc/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/postgres -l logfile start
2026-08-08 23:00:41.963 UTC [8712] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 23:00:41.963 UTC [8712] LOG: listening on IPv4 address "127.23.194.20", port 58801
2026-08-08 23:00:41.966 UTC [8712] LOG: listening on Unix socket "/tmp/.s.PGSQL.58801"
2026-08-08 23:00:41.992 UTC [8717] LOG: database system was shut down at 2026-08-08 23:00:40 UTC
2026-08-08 23:00:41.996 UTC [8712] LOG: database system is ready to accept connections
I20260808 23:00:43.042451 24328 mini_postgres.cc:96] Postgres bound to 58801
2026-08-08 23:00:43.049 UTC [8730] FATAL: database "slave" does not exist
127.23.194.20:58801 - accepting connections
I20260808 23:00:43.050040 24328 mini_ranger.cc:161] Starting Ranger...
I20260808 23:00:43.064626 24328 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 23:00:43.472751 24328 mini_ranger.cc:87] Created ranger Postgres database
I20260808 23:00:43.472831 24328 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ranger-admin
2026-08-08 23:00:43,752 [I] DB FLAVOR :POSTGRES
2026-08-08 23:00:43,752 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 23:00:43,752 [I] Checking connection..
2026-08-08 23:00:43,753 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 23:00:43,994 [I] Checking connection passed.
2026-08-08 23:00:43,995 [I] --------- Verifying version history table ---------
2026-08-08 23:00:43,995 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 23:00:44,258 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 23:00:44,259 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 23:00:44,528 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 23:00:44,529 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 23:00:44,529 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 23:00:44.758 UTC [8829] WARNING: there is no transaction in progress
2026-08-08 23:00:44,764 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 23:00:44,765 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 23:00:45,022 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 23:00:45,023 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 23:00:45,023 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 23:00:45,263 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 23:00:45,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-f0tn.c.gcp-upstream.internal','N') ;"
2026-08-08 23:00:45,775 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 23:00:46,010 [I] Table x_portal_user does not exist in database ranger
2026-08-08 23:00:46,010 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 23:00:46,271 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 23:00:46,272 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 23:00:46,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 23:00:46,764 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 23:00:46,765 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 23:00:47.673 UTC [9018] WARNING: there is no transaction in progress
2026-08-08 23:00:47.689 UTC [9018] WARNING: there is no transaction in progress
2026-08-08 23:00:47.704 UTC [9018] WARNING: there is no transaction in progress
2026-08-08 23:00:47.718 UTC [9018] WARNING: there is no transaction in progress
2026-08-08 23:00:47.731 UTC [9018] WARNING: there is no transaction in progress
2026-08-08 23:00:47.748 UTC [9018] WARNING: there is no transaction in progress
2026-08-08 23:00:47.798 UTC [9018] WARNING: there is no transaction in progress
2026-08-08 23:00:47.804 UTC [9018] WARNING: there is no transaction in progress
2026-08-08 23:00:47.816 UTC [9018] WARNING: there is no transaction in progress
2026-08-08 23:00:47.825 UTC [9018] WARNING: there is no transaction in progress
2026-08-08 23:00:47.834 UTC [9018] WARNING: there is no transaction in progress
2026-08-08 23:00:47.847 UTC [9018] WARNING: there is no transaction in progress
2026-08-08 23:00:47.856 UTC [9018] WARNING: there is no transaction in progress
2026-08-08 23:00:47.862 UTC [9018] WARNING: there is no transaction in progress
2026-08-08 23:00:47.868 UTC [9018] WARNING: there is no transaction in progress
2026-08-08 23:00:47.874 UTC [9018] WARNING: there is no transaction in progress
2026-08-08 23:00:48,423 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 23:00:48,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 23:00:48,664 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 23:00:48,665 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 23:00:48,924 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 23:00:48,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 23:00:49,167 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 23:00:49,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 23:00:49,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 23:00:49,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-f0tn.c.gcp-upstream.internal';"
2026-08-08 23:00:49,880 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 23:00:49,880 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 23:00:50,131 [I] DB_PATCHES have already been applied
I20260808 23:00:50.136988 24328 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ranger-admin:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taski6UDxc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 23:00:50.137068 24328 mini_ranger.cc:193] Using host: 127.23.194.20
I20260808 23:00:50.141760 24328 mini_ranger.cc:227] Ranger admin URL: http://127.23.194.20:33307
Aug 08, 2026 11:00:50 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 11:00:50 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 11:00:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 11:00:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 11:00:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 11:00:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 11:00:51 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 23:00:51.405197 24328 mini_ranger.cc:160] Time spent starting Ranger: real 8.355s user 0.000s sys 0.010s
Aug 08, 2026 11:01:11 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 11:01:11 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 08, 2026 11:01:11 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 11:01:11 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 11:01:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 11:01:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 11:01:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 11:01:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 11:01:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 11:01:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 11:01:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 11:01:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 11:01:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 11:01:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 11:01:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 11:01:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 11:01:11 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 11:01:12 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 23:01:13.534842 24328 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.23.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.23.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.23.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/rangerkms_127.23.194.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.23.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/rangerkms_127.23.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.194.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.23.194.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.194.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/HTTP_127.23.194.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.194.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/HTTP_127.23.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 23:01:13 dist-test-slave-f0tn krb5kdc[8658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230073, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 23:01:13.614883 24328 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 23:01:13.625483 24328 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 23:01:14.307581 24328 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 23:01:14.307677 24328 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ranger-kms
I20260808 23:01:14.307698 24328 mini_ranger_kms.cc:227] Using postgres at 127.23.194.20:58801
2026-08-08 23:01:14,973 [I] DB FLAVOR :POSTGRES
2026-08-08 23:01:14,973 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 23:01:14,974 [I] Checking connection
2026-08-08 23:01:14,974 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 23:01:15,512 [I] connection success
2026-08-08 23:01:15,512 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 23:01:15,512 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 23:01:15,513 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 23:01:15,969 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 23:01:15,970 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 23:01:15,970 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 23:01:15,970 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:58801/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 23:01:16,425 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 23:01:17.249619 24328 mini_ranger_kms.cc:310] Created kms service
I20260808 23:01:17.704298 24328 mini_ranger_kms.cc:326] Created kudu user
I20260808 23:01:17.787187 24328 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 23:01:17 dist-test-slave-f0tn krb5kdc[8658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230073, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.23.194.20@KRBTEST.COM
I20260808 23:01:18.499306 24328 mini_ranger_kms.cc:384] Added ranger policy
I20260808 23:01:18.499531 24328 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ranger-kms:/tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 23:01:18.499609 24328 mini_ranger_kms.cc:240] Using host: 127.23.194.20
I20260808 23:01:18.505468 24328 mini_ranger_kms.cc:276] Ranger KMS PID: 9301
I20260808 23:01:18.505731 24328 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.23.194.20:54013
Aug 08, 2026 11:01:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 11:01:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 11:01:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 11:01:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 11:01:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 11:01:20 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 23:01:21.420542 24328 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 7.806s user 0.010s sys 0.607s
I20260808 23:01:21.420655 24328 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 23:01:21.420727 24328 mini_ranger_kms.cc:401] 127.23.194.20:54013/kms/v1/keys
Aug 08 23:01:21 dist-test-slave-f0tn krb5kdc[8658](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786230081, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.194.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 23:01:22 dist-test-slave-f0tn krb5kdc[8658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230081, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.194.20@KRBTEST.COM for HTTP/127.23.194.20@KRBTEST.COM
Aug 08 23:01:22 dist-test-slave-f0tn krb5kdc[8658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230081, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.194.20@KRBTEST.COM for HTTP/127.23.194.20@KRBTEST.COM
Aug 08 23:01:24 dist-test-slave-f0tn krb5kdc[8658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:01:25.001295 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:34421
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:41655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:34421
--ranger_config_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.194.20:54013/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:01:25.098847 9384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:01:25.099081 9384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:01:25.099107 9384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:01:25.100289 9384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:01:25.100338 9384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:01:25.100354 9384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:01:25.100368 9384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:01:25.100380 9384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:01:25.102442 9384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:41655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/master-0/wal
--ranger_kms_url=127.23.194.20:54013/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:34421
--ranger_config_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:34421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:01:25.102698 9384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:01:25.102943 9384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 23:01:25.109295 9384 server_base.cc:1034] running on GCE node
W20260808 23:01:25.109270 9397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:01:25.109454 9400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:01:25.109318 9395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:01:25.109768 9384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:01:25.110051 9384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:01:25.111202 9384 hybrid_clock.cc:648] HybridClock initialized: now 1786230085111185 us; error 43 us; skew 500 ppm
Aug 08 23:01:25 dist-test-slave-f0tn krb5kdc[8658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230085, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:01:25.114455 9384 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:01:25.114900 9384 webserver.cc:460] Webserver started at http://127.23.194.62:34955/ using document root <none> and password file <none>
I20260808 23:01:25.115116 9384 fs_manager.cc:359] Metadata directory not provided
I20260808 23:01:25.115155 9384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:01:25.115274 9384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:01:25 dist-test-slave-f0tn krb5kdc[8658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230085, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for HTTP/127.23.194.20@KRBTEST.COM
I20260808 23:01:25.181175 9384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "32598c9bcb004b6e92b418afb43efc4c"
format_stamp: "Formatted at 2026-08-08 23:01:25 on dist-test-slave-f0tn"
server_key: "603ac3593aaf38eb2fd65c4095a02bb5"
server_key_iv: "69b83aa2ba638fd7fa8f2f8fe3d28209"
server_key_version: "kuduclusterkey@0"
I20260808 23:01:25.181648 9384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "32598c9bcb004b6e92b418afb43efc4c"
format_stamp: "Formatted at 2026-08-08 23:01:25 on dist-test-slave-f0tn"
server_key: "603ac3593aaf38eb2fd65c4095a02bb5"
server_key_iv: "69b83aa2ba638fd7fa8f2f8fe3d28209"
server_key_version: "kuduclusterkey@0"
I20260808 23:01:25.183085 9384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:01:25.200879 9409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:01:25.201508 9384 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 23:01:25.201617 9384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "32598c9bcb004b6e92b418afb43efc4c"
format_stamp: "Formatted at 2026-08-08 23:01:25 on dist-test-slave-f0tn"
server_key: "603ac3593aaf38eb2fd65c4095a02bb5"
server_key_iv: "69b83aa2ba638fd7fa8f2f8fe3d28209"
server_key_version: "kuduclusterkey@0"
I20260808 23:01:25.201709 9384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:01:25.220314 9384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:01:25.222263 9384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:01:25.222398 9384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:01:25.229235 9384 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:34421
I20260808 23:01:25.229516 9483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:34421 every 8 connection(s)
I20260808 23:01:25.229686 9384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:01:25.230603 9484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:01:25.233301 9484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c: Bootstrap starting.
I20260808 23:01:25.234143 9484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c: Neither blocks nor log segments found. Creating new log.
I20260808 23:01:25.234627 9484 log.cc:824] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c: Log is configured to *not* fsync() on all Append() calls
I20260808 23:01:25.235308 9484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c: No bootstrap required, opened a new log
I20260808 23:01:25.237802 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 9384
I20260808 23:01:25.237921 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:01:25.238147 24328 external_mini_cluster.cc:1428] Setting key 4a10e973108512c105fc766abf8a019f
I20260808 23:01:25.238287 9484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32598c9bcb004b6e92b418afb43efc4c" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34421 } }
I20260808 23:01:25.238519 9484 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:01:25.238577 9484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32598c9bcb004b6e92b418afb43efc4c, State: Initialized, Role: FOLLOWER
I20260808 23:01:25.238694 9484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32598c9bcb004b6e92b418afb43efc4c" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34421 } }
I20260808 23:01:25.238780 9484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:01:25.238811 9484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:01:25.238857 9484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:01:25.239679 9484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32598c9bcb004b6e92b418afb43efc4c" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34421 } }
I20260808 23:01:25.239785 9484 leader_election.cc:302] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32598c9bcb004b6e92b418afb43efc4c; no voters:
I20260808 23:01:25.239951 9484 leader_election.cc:288] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:01:25.240097 9493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:01:25.240227 9484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:01:25.240389 9493 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c [term 1 LEADER]: Becoming Leader. State: Replica: 32598c9bcb004b6e92b418afb43efc4c, State: Running, Role: LEADER
I20260808 23:01:25.240499 9493 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32598c9bcb004b6e92b418afb43efc4c" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34421 } }
I20260808 23:01:25.240885 9493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32598c9bcb004b6e92b418afb43efc4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32598c9bcb004b6e92b418afb43efc4c" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34421 } } }
I20260808 23:01:25.240988 9493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c [sys.catalog]: This master's current role is: LEADER
I20260808 23:01:25.241165 9493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32598c9bcb004b6e92b418afb43efc4c. Latest consensus state: current_term: 1 leader_uuid: "32598c9bcb004b6e92b418afb43efc4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32598c9bcb004b6e92b418afb43efc4c" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34421 } } }
I20260808 23:01:25.241236 9493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c [sys.catalog]: This master's current role is: LEADER
I20260808 23:01:25.242573 9484 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 23:01:25.247360 9502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:01:25.248399 9502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 23:01:25 dist-test-slave-f0tn krb5kdc[8658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:01:25.258817 9492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:01:25.238751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44262 (local address 127.23.194.62:34421)
0808 23:01:25.240704 (+ 1953us) server_negotiation.cc:206] Beginning negotiation
0808 23:01:25.240716 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:01:25.240781 (+ 65us) server_negotiation.cc:407] Connection header received
0808 23:01:25.240911 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:01:25.240919 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:01:25.241114 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:01:25.242256 (+ 1142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:01:25.242409 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:25.249214 (+ 6805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:25.250384 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:25.251018 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:25.253099 (+ 2081us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:01:25.254234 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:01:25.254240 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:01:25.254240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:01:25.254252 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:01:25.256505 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:01:25.256914 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:01:25.256917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:01:25.256917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:01:25.257105 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:01:25.257336 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:01:25.257336 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:01:25.257337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:01:25.257691 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:01:25.258295 (+ 604us) server_negotiation.cc:1035] Waiting for connection context
0808 23:01:25.258386 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 23:01:25.258508 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1879,"thread_start_us":198,"threads_started":1}
I20260808 23:01:25.259757 9502 catalog_manager.cc:1347] Generated new cluster ID: 89e2384d4a7448568887e68f4c45a298
I20260808 23:01:25.259812 9502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:01:25.272670 9502 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:01:25.273496 9502 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:01:25.282634 9502 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c: Generated new TSK 0
I20260808 23:01:25.282812 9502 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
I20260808 23:01:26.179822 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.194.20:54013/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34421
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:41655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:01:26.301997 9558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:01:26.302335 9558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:01:26.302413 9558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:01:26.303920 9558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:01:26.303982 9558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:01:26.304021 9558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:01:26.305920 9558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:41655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.23.194.20:54013/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:01:26.306169 9558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:01:26.306398 9558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:01:26.312374 9573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:01:26.312340 9570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:01:26.312666 9568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:01:26.312958 9558 server_base.cc:1034] running on GCE node
I20260808 23:01:26.313153 9558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:01:26.313467 9558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:01:26.316740 9558 hybrid_clock.cc:648] HybridClock initialized: now 1786230086316703 us; error 45 us; skew 500 ppm
Aug 08 23:01:26 dist-test-slave-f0tn krb5kdc[8658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230086, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:01:26.320070 9558 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 23:01:26.320479 9558 webserver.cc:460] Webserver started at http://127.23.194.1:40435/ using document root <none> and password file <none>
I20260808 23:01:26.320735 9558 fs_manager.cc:359] Metadata directory not provided
I20260808 23:01:26.320786 9558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:01:26.320945 9558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:01:26 dist-test-slave-f0tn krb5kdc[8658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230086, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for HTTP/127.23.194.20@KRBTEST.COM
I20260808 23:01:26.335474 9558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "611f3a94cd824d52825e9d9eb734c4ff"
format_stamp: "Formatted at 2026-08-08 23:01:26 on dist-test-slave-f0tn"
server_key: "0d4a31fa6cc539ab38acc8d80cf7412d"
server_key_iv: "cf444588e9fd802df99e9986dce044b6"
server_key_version: "kuduclusterkey@0"
I20260808 23:01:26.336027 9558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "611f3a94cd824d52825e9d9eb734c4ff"
format_stamp: "Formatted at 2026-08-08 23:01:26 on dist-test-slave-f0tn"
server_key: "0d4a31fa6cc539ab38acc8d80cf7412d"
server_key_iv: "cf444588e9fd802df99e9986dce044b6"
server_key_version: "kuduclusterkey@0"
I20260808 23:01:26.339133 9558 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:01:26.350741 9583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:01:26.350981 9558 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:01:26.351094 9558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "611f3a94cd824d52825e9d9eb734c4ff"
format_stamp: "Formatted at 2026-08-08 23:01:26 on dist-test-slave-f0tn"
server_key: "0d4a31fa6cc539ab38acc8d80cf7412d"
server_key_iv: "cf444588e9fd802df99e9986dce044b6"
server_key_version: "kuduclusterkey@0"
I20260808 23:01:26.351186 9558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:01:26.363252 9558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:01:26.365743 9558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:01:26.365963 9558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:01:26.366328 9558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:01:26.366859 9558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:01:26.366914 9558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:01:26.366948 9558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:01:26.367008 9558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:01:26.381206 9558 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:35205
I20260808 23:01:26.381340 9769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:35205 every 8 connection(s)
I20260808 23:01:26.381644 9558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:01:26.386219 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 9558
I20260808 23:01:26.386348 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:01:26.386601 24328 external_mini_cluster.cc:1428] Setting key 27601bd046ef13811286e2f226dd6b07
Aug 08 23:01:26 dist-test-slave-f0tn krb5kdc[8658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230086, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:01:26.391749 9776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:01:26.382560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34421 (local address 127.23.194.1:44649)
0808 23:01:26.382804 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 23:01:26.382847 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 23:01:26.383416 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:01:26.383664 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:01:26.383668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:01:26.383930 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:01:26.384151 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:01:26.384157 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.384845 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.384846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:01:26.385724 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:01:26.385727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.386086 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.386088 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:01:26.386226 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:01:26.386764 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:01:26.386773 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:01:26.388190 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:01:26.390240 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:01:26.390247 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:01:26.390249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:01:26.390513 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:01:26.390718 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:01:26.390718 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:01:26.390719 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:01:26.390850 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:01:26.391095 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:01:26.391102 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:01:26.391259 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 23:01:26.391361 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 23:01:26.391503 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":124,"threads_started":1}
I20260808 23:01:26.392287 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:01:26.382519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:44649 (local address 127.23.194.62:34421)
0808 23:01:26.382628 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 23:01:26.382630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:01:26.382950 (+ 320us) server_negotiation.cc:407] Connection header received
0808 23:01:26.383516 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:01:26.383518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:01:26.383533 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:01:26.383593 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:01:26.384249 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.384760 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.385805 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.386018 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.386073 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:01:26.388273 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:01:26.388274 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:01:26.388274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:01:26.388282 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:01:26.390138 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:01:26.390589 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:01:26.390590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:01:26.390591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:01:26.390653 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:01:26.390919 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:01:26.390920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:01:26.390921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:01:26.391030 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:01:26.391089 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 23:01:26.392118 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0808 23:01:26.392198 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":54,"threads_started":1}
I20260808 23:01:26.392688 9770 heartbeater.cc:347] Connected to a master server at 127.23.194.62:34421
I20260808 23:01:26.392783 9770 heartbeater.cc:464] Registering TS with master...
I20260808 23:01:26.392949 9770 heartbeater.cc:511] Master 127.23.194.62:34421 requested a full tablet report, sending...
I20260808 23:01:26.393544 9433 ts_manager.cc:194] Registered new tserver with Master: 611f3a94cd824d52825e9d9eb734c4ff (127.23.194.1:35205)
I20260808 23:01:26.394508 9433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:44649
WARNING: no policy specified for kudu/127.23.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
I20260808 23:01:26.415671 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.194.20:54013/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34421
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:41655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:01:26.563856 9781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:01:26.564147 9781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:01:26.564185 9781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:01:26.565940 9781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:01:26.566000 9781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:01:26.566039 9781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:01:26.568397 9781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:41655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.23.194.20:54013/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:01:26.568652 9781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:01:26.568986 9781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:01:26.577031 9793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:01:26.577122 9794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:01:26.577418 9799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:01:26.578485 9781 server_base.cc:1034] running on GCE node
I20260808 23:01:26.578717 9781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:01:26.579036 9781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:01:26.580214 9781 hybrid_clock.cc:648] HybridClock initialized: now 1786230086580186 us; error 41 us; skew 500 ppm
Aug 08 23:01:26 dist-test-slave-f0tn krb5kdc[8658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230086, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:01:26.584136 9781 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 23:01:26.584692 9781 webserver.cc:460] Webserver started at http://127.23.194.2:32915/ using document root <none> and password file <none>
I20260808 23:01:26.585031 9781 fs_manager.cc:359] Metadata directory not provided
I20260808 23:01:26.585105 9781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:01:26.585244 9781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:01:26 dist-test-slave-f0tn krb5kdc[8658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230086, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for HTTP/127.23.194.20@KRBTEST.COM
Aug 08 23:01:26 dist-test-slave-f0tn krb5kdc[8658](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786230086, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:01:26.604203 9781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "be9d6354a87148ee9f64931ca07acb79"
format_stamp: "Formatted at 2026-08-08 23:01:26 on dist-test-slave-f0tn"
server_key: "0329e1147493f2bd377d34cff1583e0c"
server_key_iv: "42b07ee14bd9a99e82ce210e3f52c477"
server_key_version: "kuduclusterkey@0"
I20260808 23:01:26.604715 9781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "be9d6354a87148ee9f64931ca07acb79"
format_stamp: "Formatted at 2026-08-08 23:01:26 on dist-test-slave-f0tn"
server_key: "0329e1147493f2bd377d34cff1583e0c"
server_key_iv: "42b07ee14bd9a99e82ce210e3f52c477"
server_key_version: "kuduclusterkey@0"
I20260808 23:01:26.610009 9781 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 23:01:26.622820 9808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:01:26.623063 9781 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:01:26.623148 9781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "be9d6354a87148ee9f64931ca07acb79"
format_stamp: "Formatted at 2026-08-08 23:01:26 on dist-test-slave-f0tn"
server_key: "0329e1147493f2bd377d34cff1583e0c"
server_key_iv: "42b07ee14bd9a99e82ce210e3f52c477"
server_key_version: "kuduclusterkey@0"
I20260808 23:01:26.623241 9781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:01:26.658267 9781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:01:26.661041 9781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:01:26.661269 9781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:01:26.661691 9781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:01:26.662251 9781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:01:26.662346 9781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:01:26.662415 9781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:01:26.662475 9781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:01:26.677532 9781 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:43529
I20260808 23:01:26.678131 9781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:01:26.681556 9987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:43529 every 8 connection(s)
I20260808 23:01:26.683835 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 9781
I20260808 23:01:26.683955 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:01:26.684144 24328 external_mini_cluster.cc:1428] Setting key 2903cb3e5eb9d8971d571ee5db721426
Aug 08 23:01:26 dist-test-slave-f0tn krb5kdc[8658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230086, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:01:26.688275 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:01:26.678983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:46293 (local address 127.23.194.62:34421)
0808 23:01:26.679040 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 23:01:26.679042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:01:26.679360 (+ 318us) server_negotiation.cc:407] Connection header received
0808 23:01:26.680050 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:01:26.680052 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:01:26.680068 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:01:26.680120 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:01:26.680892 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.681427 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.682662 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.682892 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.682949 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:01:26.685456 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:01:26.685457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:01:26.685458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:01:26.685466 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:01:26.686795 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:01:26.687277 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:01:26.687278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:01:26.687278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:01:26.687337 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:01:26.687586 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:01:26.687587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:01:26.687587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:01:26.687712 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:01:26.687750 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 23:01:26.688122 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 23:01:26.688207 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 23:01:26.688730 9990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:01:26.678962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34421 (local address 127.23.194.2:46293)
0808 23:01:26.679216 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 23:01:26.679263 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 23:01:26.679939 (+ 676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:01:26.680191 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:01:26.680196 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:01:26.680542 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:01:26.680795 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:01:26.680801 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.681539 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.681540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:01:26.682578 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:01:26.682581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.682966 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.682967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:01:26.683103 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:01:26.683715 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:01:26.683723 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:01:26.685350 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:01:26.686891 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:01:26.686899 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:01:26.686902 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:01:26.687191 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:01:26.687400 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:01:26.687401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:01:26.687401 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:01:26.687520 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:01:26.687793 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:01:26.687799 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:01:26.687952 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 23:01:26.688363 (+ 411us) client_negotiation.cc:240] Negotiation successful
0808 23:01:26.688521 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":114,"threads_started":1}
I20260808 23:01:26.689414 9988 heartbeater.cc:347] Connected to a master server at 127.23.194.62:34421
I20260808 23:01:26.689520 9988 heartbeater.cc:464] Registering TS with master...
I20260808 23:01:26.689700 9988 heartbeater.cc:511] Master 127.23.194.62:34421 requested a full tablet report, sending...
I20260808 23:01:26.690078 9433 ts_manager.cc:194] Registered new tserver with Master: be9d6354a87148ee9f64931ca07acb79 (127.23.194.2:43529)
I20260808 23:01:26.690551 9433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:46293
WARNING: no policy specified for kudu/127.23.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
I20260808 23:01:26.716513 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.194.20:54013/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34421
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:41655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:01:26.852218 9994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:01:26.852422 9994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:01:26.852452 9994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:01:26.853621 9994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:01:26.853662 9994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:01:26.853698 9994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:01:26.855293 9994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:41655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.23.194.20:54013/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:01:26.855624 9994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:01:26.855849 9994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:01:26.862067 10002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:01:26.862381 10001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:01:26.862459 10005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:01:26.862938 9994 server_base.cc:1034] running on GCE node
I20260808 23:01:26.863130 9994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:01:26.863346 9994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:01:26.864490 9994 hybrid_clock.cc:648] HybridClock initialized: now 1786230086864466 us; error 49 us; skew 500 ppm
Aug 08 23:01:26 dist-test-slave-f0tn krb5kdc[8658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230086, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:01:26.867445 9994 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 23:01:26.867832 9994 webserver.cc:460] Webserver started at http://127.23.194.3:45213/ using document root <none> and password file <none>
I20260808 23:01:26.868065 9994 fs_manager.cc:359] Metadata directory not provided
I20260808 23:01:26.868152 9994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:01:26.868307 9994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:01:26 dist-test-slave-f0tn krb5kdc[8658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230086, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for HTTP/127.23.194.20@KRBTEST.COM
I20260808 23:01:26.880309 9994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "99a6a118f69548838610e5ca0459d970"
format_stamp: "Formatted at 2026-08-08 23:01:26 on dist-test-slave-f0tn"
server_key: "67a37192b619cb4deb974d719177e8a5"
server_key_iv: "2b62419b05ff7da5b1cb3f1033c187ce"
server_key_version: "kuduclusterkey@0"
I20260808 23:01:26.880852 9994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "99a6a118f69548838610e5ca0459d970"
format_stamp: "Formatted at 2026-08-08 23:01:26 on dist-test-slave-f0tn"
server_key: "67a37192b619cb4deb974d719177e8a5"
server_key_iv: "2b62419b05ff7da5b1cb3f1033c187ce"
server_key_version: "kuduclusterkey@0"
I20260808 23:01:26.885892 9994 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260808 23:01:26.905006 10015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:01:26.905504 9994 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260808 23:01:26.905632 9994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "99a6a118f69548838610e5ca0459d970"
format_stamp: "Formatted at 2026-08-08 23:01:26 on dist-test-slave-f0tn"
server_key: "67a37192b619cb4deb974d719177e8a5"
server_key_iv: "2b62419b05ff7da5b1cb3f1033c187ce"
server_key_version: "kuduclusterkey@0"
I20260808 23:01:26.905721 9994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:01:26.928881 9994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:01:26.931237 9994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:01:26.931468 9994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:01:26.931849 9994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:01:26.932317 9994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:01:26.932420 9994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:01:26.932495 9994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:01:26.932569 9994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:01:26.951891 9994 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:46731
I20260808 23:01:26.952128 10192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:46731 every 8 connection(s)
I20260808 23:01:26.952553 9994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 23:01:26.955065 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 9994
I20260808 23:01:26.955241 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:01:26.955464 24328 external_mini_cluster.cc:1428] Setting key 4d895bb89c33e167c1bd675bbb5dc28f
Aug 08 23:01:26 dist-test-slave-f0tn krb5kdc[8658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230086, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:01:26.964365 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:01:26.954219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:49747 (local address 127.23.194.62:34421)
0808 23:01:26.954268 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 23:01:26.954270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:01:26.954283 (+ 13us) server_negotiation.cc:407] Connection header received
0808 23:01:26.955126 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:01:26.955128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:01:26.955142 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:01:26.955174 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:01:26.956238 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.957034 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.958595 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.958865 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.958909 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:01:26.961683 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:01:26.961685 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:01:26.961685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:01:26.961692 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:01:26.963111 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:01:26.963469 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:01:26.963470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:01:26.963470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:01:26.963525 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:01:26.963746 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:01:26.963746 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:01:26.963747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:01:26.963866 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:01:26.963921 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 23:01:26.964228 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 23:01:26.964300 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 23:01:26.964859 10196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:01:26.953518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34421 (local address 127.23.194.3:49747)
0808 23:01:26.953954 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0808 23:01:26.953999 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 23:01:26.954985 (+ 986us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:01:26.955375 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:01:26.955379 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:01:26.955706 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:01:26.956123 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:01:26.956130 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.957143 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.957144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:01:26.958484 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:01:26.958488 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.959001 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.959002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:01:26.959157 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:01:26.959818 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:01:26.959828 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:01:26.961563 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:01:26.963187 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:01:26.963194 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:01:26.963196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:01:26.963403 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:01:26.963581 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:01:26.963581 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:01:26.963582 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:01:26.963691 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:01:26.963920 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:01:26.963927 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:01:26.964068 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 23:01:26.964461 (+ 393us) client_negotiation.cc:240] Negotiation successful
0808 23:01:26.964630 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":180,"threads_started":1}
I20260808 23:01:26.965350 10194 heartbeater.cc:347] Connected to a master server at 127.23.194.62:34421
I20260808 23:01:26.965474 10194 heartbeater.cc:464] Registering TS with master...
I20260808 23:01:26.965662 10194 heartbeater.cc:511] Master 127.23.194.62:34421 requested a full tablet report, sending...
I20260808 23:01:26.966028 9433 ts_manager.cc:194] Registered new tserver with Master: 99a6a118f69548838610e5ca0459d970 (127.23.194.3:46731)
I20260808 23:01:26.966478 9433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:49747
I20260808 23:01:26.968500 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:01:26.978420 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:01:26.971747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44276 (local address 127.23.194.62:34421)
0808 23:01:26.971812 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 23:01:26.971814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:01:26.972041 (+ 227us) server_negotiation.cc:407] Connection header received
0808 23:01:26.972065 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:01:26.972066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:01:26.972075 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:01:26.972146 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:01:26.972896 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.973421 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.974423 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.974608 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.974671 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:01:26.976022 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:01:26.976023 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:01:26.976023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:01:26.976029 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:01:26.977263 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:01:26.977699 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:01:26.977700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:01:26.977700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:01:26.977761 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:01:26.977961 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:01:26.977961 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:01:26.977961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:01:26.978082 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:01:26.978155 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 23:01:26.978297 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 23:01:26.978356 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 23:01:26.980263 9433 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:01:26.982118 9433 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:01:26.989676 10206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:01:26.984952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:43529 (local address 127.0.0.1:43346)
0808 23:01:26.985848 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0808 23:01:26.985887 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 23:01:26.985944 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:01:26.986162 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:01:26.986165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:01:26.986166 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:01:26.986218 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:01:26.986219 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.987694 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.987696 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:01:26.988513 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:01:26.988516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.989336 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.989337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:01:26.989450 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:01:26.989468 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 23:01:26.989537 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 23:01:26.989595 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":152,"threads_started":1}
I20260808 23:01:26.989756 10211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:01:26.985352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43346 (local address 127.23.194.2:43529)
0808 23:01:26.985857 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0808 23:01:26.985861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:01:26.985932 (+ 71us) server_negotiation.cc:407] Connection header received
0808 23:01:26.986048 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:01:26.986051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:01:26.986068 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:01:26.986152 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:01:26.986251 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.987607 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.988665 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.989255 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.989322 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:01:26.989540 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 23:01:26.989571 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 23:01:26.989610 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":131,"threads_started":1}
I20260808 23:01:26.990500 9891 tablet_service.cc:1467] Processing CreateTablet for tablet bd812c2de7ed45b4953c51c2cd61323d (DEFAULT_TABLE table=test-table [id=f531e611a5b54407875e50846d5f6c6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:01:26.990936 9891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd812c2de7ed45b4953c51c2cd61323d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:01:26.993983 10206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:01:26.984981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:35205 (local address 127.0.0.1:57752)
0808 23:01:26.989856 (+ 4875us) negotiation.cc:107] Waiting for socket to connect
0808 23:01:26.989866 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:01:26.989939 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:01:26.990186 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:01:26.990188 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:01:26.990189 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:01:26.990218 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:01:26.990219 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.991505 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.991506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:01:26.992510 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:01:26.992513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.993760 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.993761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:01:26.993821 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:01:26.993825 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:01:26.993880 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 23:01:26.993920 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4865,"thread_start_us":816,"threads_started":1}
I20260808 23:01:26.994076 10209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:01:26.985201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57752 (local address 127.23.194.1:35205)
0808 23:01:26.985596 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0808 23:01:26.985600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:01:26.989912 (+ 4312us) server_negotiation.cc:407] Connection header received
0808 23:01:26.990018 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:01:26.990021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:01:26.990039 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:01:26.990090 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:01:26.990292 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.991409 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.992621 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.993503 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.993665 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:01:26.993734 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 23:01:26.993912 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 23:01:26.993966 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":65,"threads_started":1}
I20260808 23:01:26.994891 9661 tablet_service.cc:1467] Processing CreateTablet for tablet bd812c2de7ed45b4953c51c2cd61323d (DEFAULT_TABLE table=test-table [id=f531e611a5b54407875e50846d5f6c6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:01:26.995234 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd812c2de7ed45b4953c51c2cd61323d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:01:26.998070 10213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:01:26.985009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:46731 (local address 127.0.0.1:46120)
0808 23:01:26.993519 (+ 8510us) negotiation.cc:107] Waiting for socket to connect
0808 23:01:26.993530 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:01:26.993593 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:01:26.993970 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:01:26.993970 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:01:26.993972 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:01:26.994016 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:01:26.994018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.995258 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.995259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:01:26.996336 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:01:26.996339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.997846 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.997847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:01:26.997911 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:01:26.997915 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:01:26.997965 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 23:01:26.998002 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8471,"thread_start_us":730,"threads_started":1}
I20260808 23:01:26.998413 10210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:01:26.985210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46120 (local address 127.23.194.3:46731)
0808 23:01:26.985512 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0808 23:01:26.985518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:01:26.993657 (+ 8139us) server_negotiation.cc:407] Connection header received
0808 23:01:26.993736 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:01:26.993738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:01:26.993756 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:01:26.993826 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:01:26.994097 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.995157 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.996522 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:26.997629 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:26.997702 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:01:26.997807 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 23:01:26.998290 (+ 483us) server_negotiation.cc:299] Negotiation successful
0808 23:01:26.998333 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":157,"threads_started":1}
I20260808 23:01:26.999419 10217 tablet_bootstrap.cc:492] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff: Bootstrap starting.
I20260808 23:01:27.000717 10217 tablet_bootstrap.cc:654] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff: Neither blocks nor log segments found. Creating new log.
I20260808 23:01:27.001240 10215 tablet_bootstrap.cc:492] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79: Bootstrap starting.
I20260808 23:01:27.001400 10217 log.cc:824] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff: Log is configured to *not* fsync() on all Append() calls
I20260808 23:01:27.002229 10086 tablet_service.cc:1467] Processing CreateTablet for tablet bd812c2de7ed45b4953c51c2cd61323d (DEFAULT_TABLE table=test-table [id=f531e611a5b54407875e50846d5f6c6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:01:27.002601 10086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd812c2de7ed45b4953c51c2cd61323d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:01:27.002974 10215 tablet_bootstrap.cc:654] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79: Neither blocks nor log segments found. Creating new log.
I20260808 23:01:27.003543 10215 log.cc:824] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79: Log is configured to *not* fsync() on all Append() calls
I20260808 23:01:27.004426 10215 tablet_bootstrap.cc:492] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79: No bootstrap required, opened a new log
I20260808 23:01:27.004541 10215 ts_tablet_manager.cc:1397] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260808 23:01:27.004700 10217 tablet_bootstrap.cc:492] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff: No bootstrap required, opened a new log
I20260808 23:01:27.004772 10217 ts_tablet_manager.cc:1397] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 23:01:27.008250 10215 raft_consensus.cc:348] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9d6354a87148ee9f64931ca07acb79" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43529 } } peers { permanent_uuid: "99a6a118f69548838610e5ca0459d970" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 46731 } } peers { permanent_uuid: "611f3a94cd824d52825e9d9eb734c4ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 35205 } }
I20260808 23:01:27.008458 10215 raft_consensus.cc:374] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:01:27.008499 10215 raft_consensus.cc:729] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9d6354a87148ee9f64931ca07acb79, State: Initialized, Role: FOLLOWER
I20260808 23:01:27.008643 10215 consensus_queue.cc:260] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9d6354a87148ee9f64931ca07acb79" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43529 } } peers { permanent_uuid: "99a6a118f69548838610e5ca0459d970" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 46731 } } peers { permanent_uuid: "611f3a94cd824d52825e9d9eb734c4ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 35205 } }
I20260808 23:01:27.008685 10217 raft_consensus.cc:348] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9d6354a87148ee9f64931ca07acb79" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43529 } } peers { permanent_uuid: "99a6a118f69548838610e5ca0459d970" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 46731 } } peers { permanent_uuid: "611f3a94cd824d52825e9d9eb734c4ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 35205 } }
I20260808 23:01:27.008829 10217 raft_consensus.cc:374] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:01:27.008874 10217 raft_consensus.cc:729] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611f3a94cd824d52825e9d9eb734c4ff, State: Initialized, Role: FOLLOWER
I20260808 23:01:27.008942 10215 ts_tablet_manager.cc:1428] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260808 23:01:27.009057 9988 heartbeater.cc:503] Master 127.23.194.62:34421 was elected leader, sending a full tablet report...
I20260808 23:01:27.008988 10217 consensus_queue.cc:260] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9d6354a87148ee9f64931ca07acb79" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43529 } } peers { permanent_uuid: "99a6a118f69548838610e5ca0459d970" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 46731 } } peers { permanent_uuid: "611f3a94cd824d52825e9d9eb734c4ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 35205 } }
I20260808 23:01:27.009619 9770 heartbeater.cc:503] Master 127.23.194.62:34421 was elected leader, sending a full tablet report...
I20260808 23:01:27.009958 10217 ts_tablet_manager.cc:1428] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260808 23:01:27.010428 10226 tablet_bootstrap.cc:492] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970: Bootstrap starting.
I20260808 23:01:27.011790 10226 tablet_bootstrap.cc:654] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970: Neither blocks nor log segments found. Creating new log.
I20260808 23:01:27.012271 10226 log.cc:824] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970: Log is configured to *not* fsync() on all Append() calls
I20260808 23:01:27.013221 10226 tablet_bootstrap.cc:492] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970: No bootstrap required, opened a new log
I20260808 23:01:27.013305 10226 ts_tablet_manager.cc:1397] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 23:01:27.016767 10226 raft_consensus.cc:348] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9d6354a87148ee9f64931ca07acb79" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43529 } } peers { permanent_uuid: "99a6a118f69548838610e5ca0459d970" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 46731 } } peers { permanent_uuid: "611f3a94cd824d52825e9d9eb734c4ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 35205 } }
I20260808 23:01:27.016937 10226 raft_consensus.cc:374] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:01:27.016984 10226 raft_consensus.cc:729] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a6a118f69548838610e5ca0459d970, State: Initialized, Role: FOLLOWER
I20260808 23:01:27.017105 10226 consensus_queue.cc:260] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9d6354a87148ee9f64931ca07acb79" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43529 } } peers { permanent_uuid: "99a6a118f69548838610e5ca0459d970" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 46731 } } peers { permanent_uuid: "611f3a94cd824d52825e9d9eb734c4ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 35205 } }
I20260808 23:01:27.017455 10226 ts_tablet_manager.cc:1428] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260808 23:01:27.017592 10194 heartbeater.cc:503] Master 127.23.194.62:34421 was elected leader, sending a full tablet report...
I20260808 23:01:27.035328 10228 raft_consensus.cc:482] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:01:27.035456 10228 raft_consensus.cc:504] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9d6354a87148ee9f64931ca07acb79" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43529 } } peers { permanent_uuid: "99a6a118f69548838610e5ca0459d970" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 46731 } } peers { permanent_uuid: "611f3a94cd824d52825e9d9eb734c4ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 35205 } }
I20260808 23:01:27.035802 10228 leader_election.cc:288] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9d6354a87148ee9f64931ca07acb79 (127.23.194.2:43529), 99a6a118f69548838610e5ca0459d970 (127.23.194.3:46731)
I20260808 23:01:27.040503 10210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:01:27.036062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:58509 (local address 127.23.194.3:46731)
0808 23:01:27.036113 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 23:01:27.036115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:01:27.036407 (+ 292us) server_negotiation.cc:407] Connection header received
0808 23:01:27.036699 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:01:27.036701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:01:27.036713 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:01:27.036757 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:01:27.037142 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:27.037901 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:27.039442 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:27.040035 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:27.040082 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:01:27.040092 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 23:01:27.040390 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 23:01:27.040432 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 23:01:27.040787 10232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:01:27.035955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:46731 (local address 127.23.194.1:58509)
0808 23:01:27.036327 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0808 23:01:27.036340 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 23:01:27.036473 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:01:27.036803 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:01:27.036804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:01:27.036807 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:01:27.037000 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:01:27.037003 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:01:27.038043 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:01:27.038044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:01:27.039344 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:01:27.039347 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:01:27.040162 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:01:27.040162 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:01:27.040302 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:01:27.040307 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:01:27.040616 (+ 309us) client_negotiation.cc:240] Negotiation successful
0808 23:01:27.040663 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":263,"threads_started":1}
I20260808 23:01:27.041193 10102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd812c2de7ed45b4953c51c2cd61323d" candidate_uuid: "611f3a94cd824d52825e9d9eb734c4ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a6a118f69548838610e5ca0459d970" is_pre_election: true
I20260808 23:01:27.041424 10102 raft_consensus.cc:2454] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611f3a94cd824d52825e9d9eb734c4ff in term 0.
I20260808 23:01:27.041716 9584 leader_election.cc:302] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611f3a94cd824d52825e9d9eb734c4ff, 99a6a118f69548838610e5ca0459d970; no voters:
I20260808 23:01:27.041879 10228 raft_consensus.cc:2781] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:01:27.041927 10228 raft_consensus.cc:482] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:01:27.041949 10228 raft_consensus.cc:3037] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:01:27.043045 10228 raft_consensus.cc:504] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9d6354a87148ee9f64931ca07acb79" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43529 } } peers { permanent_uuid: "99a6a118f69548838610e5ca0459d970" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 46731 } } peers { permanent_uuid: "611f3a94cd824d52825e9d9eb734c4ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 35205 } }
I20260808 23:01:27.043190 10228 leader_election.cc:288] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff [CANDIDATE]: Term 1 election: Requested vote from peers be9d6354a87148ee9f64931ca07acb79 (127.23.194.2:43529), 99a6a118f69548838610e5ca0459d970 (127.23.194.3:46731)
I20260808 23:01:27.043388 10102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd812c2de7ed45b4953c51c2cd61323d" candidate_uuid: "611f3a94cd824d52825e9d9eb734c4ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a6a118f69548838610e5ca0459d970"
I20260808 23:01:27.043455 10102 raft_consensus.cc:3037] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:01:27.044191 10211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:01:27.036093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:52291 (local address 127.23.194.2:43529)
0808 23:01:27.036181 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 23:01:27.036183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:01:27.036447 (+ 264us) server_negotiation.cc:407] Connection header received
0808 23:01:27.036576 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:01:27.036577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:01:27.036590 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:01:27.036640 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:01:27.039917 (+ 3277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:27.040650 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:27.043038 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:27.043573 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:27.043629 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:01:27.043640 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 23:01:27.044068 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 23:01:27.044117 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:01:27.044525 10102 raft_consensus.cc:2454] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611f3a94cd824d52825e9d9eb734c4ff in term 1.
I20260808 23:01:27.044705 9584 leader_election.cc:302] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611f3a94cd824d52825e9d9eb734c4ff, 99a6a118f69548838610e5ca0459d970; no voters:
I20260808 23:01:27.044787 10228 raft_consensus.cc:2781] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:01:27.044872 10234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:01:27.035967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:43529 (local address 127.23.194.1:52291)
0808 23:01:27.036386 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0808 23:01:27.036395 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:01:27.036473 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:01:27.036647 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:01:27.036648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:01:27.036650 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:01:27.039793 (+ 3143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:01:27.039797 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:01:27.041166 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:01:27.041167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:01:27.042900 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:01:27.042904 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:01:27.043801 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:01:27.043802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:01:27.043951 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:01:27.043956 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:01:27.044479 (+ 523us) client_negotiation.cc:240] Negotiation successful
0808 23:01:27.044528 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":1,"thread_start_us":320,"threads_started":1}
I20260808 23:01:27.044943 10228 raft_consensus.cc:686] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff [term 1 LEADER]: Becoming Leader. State: Replica: 611f3a94cd824d52825e9d9eb734c4ff, State: Running, Role: LEADER
I20260808 23:01:27.045198 10228 consensus_queue.cc:237] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9d6354a87148ee9f64931ca07acb79" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43529 } } peers { permanent_uuid: "99a6a118f69548838610e5ca0459d970" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 46731 } } peers { permanent_uuid: "611f3a94cd824d52825e9d9eb734c4ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 35205 } }
I20260808 23:01:27.045660 9913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd812c2de7ed45b4953c51c2cd61323d" candidate_uuid: "611f3a94cd824d52825e9d9eb734c4ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9d6354a87148ee9f64931ca07acb79"
I20260808 23:01:27.045795 9913 raft_consensus.cc:3037] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:01:27.046116 9431 catalog_manager.cc:5495] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 611f3a94cd824d52825e9d9eb734c4ff (127.23.194.1). New cstate: current_term: 1 leader_uuid: "611f3a94cd824d52825e9d9eb734c4ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9d6354a87148ee9f64931ca07acb79" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99a6a118f69548838610e5ca0459d970" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 46731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611f3a94cd824d52825e9d9eb734c4ff" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 35205 } health_report { overall_health: HEALTHY } } }
I20260808 23:01:27.047178 9913 raft_consensus.cc:2454] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611f3a94cd824d52825e9d9eb734c4ff in term 1.
I20260808 23:01:27.047508 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd812c2de7ed45b4953c51c2cd61323d" candidate_uuid: "611f3a94cd824d52825e9d9eb734c4ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9d6354a87148ee9f64931ca07acb79" is_pre_election: true
I20260808 23:01:27.047621 9915 raft_consensus.cc:2362] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 611f3a94cd824d52825e9d9eb734c4ff in term 1. Re-sending same reply.
I20260808 23:01:27.054981 10209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:01:27.052566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57768 (local address 127.23.194.1:35205)
0808 23:01:27.052610 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 23:01:27.052612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:01:27.052657 (+ 45us) server_negotiation.cc:407] Connection header received
0808 23:01:27.052731 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:01:27.052732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:01:27.052750 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:01:27.052814 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:01:27.052918 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:27.053414 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:27.054168 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:01:27.054362 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:01:27.054417 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:01:27.054556 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:01:27.054656 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:01:27.054726 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 23:01:27.054763 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 23:01:27.054785 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 23:01:27.056833 10102 raft_consensus.cc:1261] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970 [term 1 FOLLOWER]: Refusing update from remote peer 611f3a94cd824d52825e9d9eb734c4ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:01:27.056849 9915 raft_consensus.cc:1261] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79 [term 1 FOLLOWER]: Refusing update from remote peer 611f3a94cd824d52825e9d9eb734c4ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:01:27.057149 10228 consensus_queue.cc:1035] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a6a118f69548838610e5ca0459d970" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 46731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:01:27.057415 10237 consensus_queue.cc:1035] T bd812c2de7ed45b4953c51c2cd61323d P 611f3a94cd824d52825e9d9eb734c4ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9d6354a87148ee9f64931ca07acb79" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:01:27.061620 10240 mvcc.cc:205] Tried to move back new op lower bound from 7316398436582264832 to 7316398436537974784. Current Snapshot: MvccSnapshot[applied={T|T < 7316398436582264832}]
I20260808 23:01:27.062412 10245 mvcc.cc:205] Tried to move back new op lower bound from 7316398436582264832 to 7316398436537974784. Current Snapshot: MvccSnapshot[applied={T|T < 7316398436582264832}]
I20260808 23:01:27.064452 10247 mvcc.cc:205] Tried to move back new op lower bound from 7316398436582264832 to 7316398436537974784. Current Snapshot: MvccSnapshot[applied={T|T < 7316398436582264832}]
I20260808 23:01:27.066820 9433 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:01:27.066910 9433 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:01:27.067675 9433 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32598c9bcb004b6e92b418afb43efc4c: Sending DeleteTablet for 3 replicas of tablet bd812c2de7ed45b4953c51c2cd61323d
I20260808 23:01:27.068358 9891 tablet_service.cc:1514] Processing DeleteTablet for tablet bd812c2de7ed45b4953c51c2cd61323d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:01:27 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:43346
I20260808 23:01:27.068621 10086 tablet_service.cc:1514] Processing DeleteTablet for tablet bd812c2de7ed45b4953c51c2cd61323d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:01:27 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:46120
I20260808 23:01:27.068882 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet bd812c2de7ed45b4953c51c2cd61323d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:01:27 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:57752
I20260808 23:01:27.069748 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 9558
I20260808 23:01:27.080986 10262 tablet_replica.cc:330] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970: stopping tablet replica
I20260808 23:01:27.081194 10262 raft_consensus.cc:2229] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:01:27.081440 10262 raft_consensus.cc:2258] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:01:27.082473 10262 ts_tablet_manager.cc:1905] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:01:27.085709 10261 tablet_replica.cc:330] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79: stopping tablet replica
I20260808 23:01:27.085840 10261 raft_consensus.cc:2229] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:01:27.085954 10261 raft_consensus.cc:2258] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:01:27.086462 10261 ts_tablet_manager.cc:1905] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:01:27.087991 10261 ts_tablet_manager.cc:1918] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:01:27.088058 10261 log.cc:1196] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/bd812c2de7ed45b4953c51c2cd61323d
I20260808 23:01:27.088653 10261 ts_tablet_manager.cc:1939] T bd812c2de7ed45b4953c51c2cd61323d P be9d6354a87148ee9f64931ca07acb79: Deleting consensus metadata
I20260808 23:01:27.089145 9413 catalog_manager.cc:4841] TS be9d6354a87148ee9f64931ca07acb79 (127.23.194.2:43529): tablet bd812c2de7ed45b4953c51c2cd61323d (table test-table [id=f531e611a5b54407875e50846d5f6c6d]) successfully deleted
W20260808 23:01:27.089967 9412 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.194.1:35205 (error 108)
I20260808 23:01:27.090547 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 9781
I20260808 23:01:27.095829 10262 ts_tablet_manager.cc:1918] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:01:27.095925 10262 log.cc:1196] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/bd812c2de7ed45b4953c51c2cd61323d
I20260808 23:01:27.096276 10262 ts_tablet_manager.cc:1939] T bd812c2de7ed45b4953c51c2cd61323d P 99a6a118f69548838610e5ca0459d970: Deleting consensus metadata
I20260808 23:01:27.096792 9410 catalog_manager.cc:4841] TS 99a6a118f69548838610e5ca0459d970 (127.23.194.3:46731): tablet bd812c2de7ed45b4953c51c2cd61323d (table test-table [id=f531e611a5b54407875e50846d5f6c6d]) successfully deleted
I20260808 23:01:27.096913 10213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:01:27.094874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:35205 (local address 127.0.0.1:57770)
0808 23:01:27.096835 (+ 1961us) negotiation.cc:107] Waiting for socket to connect
0808 23:01:27.096884 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.1:35205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1934}
W20260808 23:01:27.097026 9412 catalog_manager.cc:4584] TS 611f3a94cd824d52825e9d9eb734c4ff (127.23.194.1:35205): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bd812c2de7ed45b4953c51c2cd61323d: Network error: Client connection negotiation failed: client connection to 127.23.194.1:35205: connect: Connection refused (error 111)
I20260808 23:01:27.101346 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 9994
I20260808 23:01:27.128453 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 9384
I20260808 23:01:27.146373 24328 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 23:01:27.492095 24328 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 23:01:27.492213 24328 mini_ranger.cc:66] Stopping Ranger...
I20260808 23:01:27.836095 24328 mini_ranger.cc:68] Stopped Ranger
2026-08-08 23:01:27.836 UTC [8712] LOG: received smart shutdown request
2026-08-08 23:01:27.840 UTC [8712] LOG: background worker "logical replication launcher" (PID 8727) exited with exit code 1
2026-08-08 23:01:27.840 UTC [8718] LOG: shutting down
2026-08-08 23:01:27.975 UTC [8712] LOG: database system is shut down
2026-08-08T23:01:27Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49591 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:01:28 dist-test-slave-f0tn krb5kdc[10278](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:01:28 dist-test-slave-f0tn krb5kdc[10278](info): set up 2 sockets
Aug 08 23:01:28 dist-test-slave-f0tn krb5kdc[10278](info): commencing operation
krb5kdc: starting...
W20260808 23:01:29.211338 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:01:29 dist-test-slave-f0tn krb5kdc[10278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:01:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:01:29Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.23.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.23.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.23.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/rangeradmin_127.23.194.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.23.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/rangeradmin_127.23.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.23.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.23.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.23.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/rangerlookup_127.23.194.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.23.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/rangerlookup_127.23.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.23.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/HTTP_127.23.194.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/HTTP_127.23.194.20@KRBTEST.COM.keytab.
I20260808 23:01:29.365545 24328 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taski6UDxc/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/postgres -l logfile start
2026-08-08 23:01:31.523 UTC [10326] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 23:01:31.523 UTC [10326] LOG: listening on IPv4 address "127.23.194.20", port 41121
2026-08-08 23:01:31.526 UTC [10326] LOG: listening on Unix socket "/tmp/.s.PGSQL.41121"
2026-08-08 23:01:31.547 UTC [10331] LOG: database system was shut down at 2026-08-08 23:01:30 UTC
2026-08-08 23:01:31.551 UTC [10326] LOG: database system is ready to accept connections
I20260808 23:01:32.606390 24328 mini_postgres.cc:96] Postgres bound to 41121
2026-08-08 23:01:32.613 UTC [10342] FATAL: database "slave" does not exist
127.23.194.20:41121 - accepting connections
I20260808 23:01:32.613658 24328 mini_ranger.cc:161] Starting Ranger...
I20260808 23:01:32.628777 24328 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 23:01:33.017372 24328 mini_ranger.cc:87] Created ranger Postgres database
I20260808 23:01:33.017479 24328 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ranger-admin
2026-08-08 23:01:33,293 [I] DB FLAVOR :POSTGRES
2026-08-08 23:01:33,294 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 23:01:33,294 [I] Checking connection..
2026-08-08 23:01:33,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 23:01:33,544 [I] Checking connection passed.
2026-08-08 23:01:33,544 [I] --------- Verifying version history table ---------
2026-08-08 23:01:33,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 23:01:33,804 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 23:01:33,805 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 23:01:34,043 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 23:01:34,043 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 23:01:34,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 23:01:34.299 UTC [10438] WARNING: there is no transaction in progress
2026-08-08 23:01:34,307 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 23:01:34,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 23:01:34,564 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 23:01:34,564 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 23:01:34,565 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 23:01:34,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 23:01:35,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-f0tn.c.gcp-upstream.internal','N') ;"
2026-08-08 23:01:35,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 23:01:35,572 [I] Table x_portal_user does not exist in database ranger
2026-08-08 23:01:35,573 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 23:01:35,816 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 23:01:35,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 23:01:36,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 23:01:36,344 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 23:01:36,345 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 23:01:37.252 UTC [10614] WARNING: there is no transaction in progress
2026-08-08 23:01:37.266 UTC [10614] WARNING: there is no transaction in progress
2026-08-08 23:01:37.281 UTC [10614] WARNING: there is no transaction in progress
2026-08-08 23:01:37.295 UTC [10614] WARNING: there is no transaction in progress
2026-08-08 23:01:37.313 UTC [10614] WARNING: there is no transaction in progress
2026-08-08 23:01:37.332 UTC [10614] WARNING: there is no transaction in progress
2026-08-08 23:01:37.388 UTC [10614] WARNING: there is no transaction in progress
2026-08-08 23:01:37.394 UTC [10614] WARNING: there is no transaction in progress
2026-08-08 23:01:37.406 UTC [10614] WARNING: there is no transaction in progress
2026-08-08 23:01:37.414 UTC [10614] WARNING: there is no transaction in progress
2026-08-08 23:01:37.423 UTC [10614] WARNING: there is no transaction in progress
2026-08-08 23:01:37.437 UTC [10614] WARNING: there is no transaction in progress
2026-08-08 23:01:37.446 UTC [10614] WARNING: there is no transaction in progress
2026-08-08 23:01:37.452 UTC [10614] WARNING: there is no transaction in progress
2026-08-08 23:01:37.457 UTC [10614] WARNING: there is no transaction in progress
2026-08-08 23:01:37.463 UTC [10614] WARNING: there is no transaction in progress
2026-08-08 23:01:38,055 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 23:01:38,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 23:01:38,298 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 23:01:38,298 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 23:01:38,571 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 23:01:38,571 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 23:01:38,838 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 23:01:38,839 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 23:01:39,098 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 23:01:39,348 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-f0tn.c.gcp-upstream.internal';"
2026-08-08 23:01:39,607 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 23:01:39,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 23:01:39,877 [I] DB_PATCHES have already been applied
I20260808 23:01:39.882962 24328 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ranger-admin:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taski6UDxc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 23:01:39.883033 24328 mini_ranger.cc:193] Using host: 127.23.194.20
I20260808 23:01:39.886845 24328 mini_ranger.cc:227] Ranger admin URL: http://127.23.194.20:37981
Aug 08, 2026 11:01:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 11:01:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 11:01:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 11:01:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 11:01:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 11:01:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 11:01:40 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 23:01:41.124454 24328 mini_ranger.cc:160] Time spent starting Ranger: real 8.511s user 0.004s sys 0.005s
Aug 08, 2026 11:02:00 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 11:02:00 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 11:02:00 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 11:02:00 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 11:02:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 11:02:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 11:02:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 11:02:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 11:02:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 11:02:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 11:02:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 11:02:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 11:02:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 11:02:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 11:02:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 11:02:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 11:02:00 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 11:02:01 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 23:02:02.451675 24328 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.23.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.23.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.23.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/rangerkms_127.23.194.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.23.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/rangerkms_127.23.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.194.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.23.194.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.194.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/HTTP_127.23.194.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.194.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/HTTP_127.23.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 23:02:02 dist-test-slave-f0tn krb5kdc[10278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230122, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 23:02:02.534883 24328 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 23:02:02.545014 24328 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 23:02:03.275040 24328 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 23:02:03.275125 24328 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ranger-kms
I20260808 23:02:03.275141 24328 mini_ranger_kms.cc:227] Using postgres at 127.23.194.20:41121
2026-08-08 23:02:03,432 [I] DB FLAVOR :POSTGRES
2026-08-08 23:02:03,433 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 23:02:03,433 [I] Checking connection
2026-08-08 23:02:03,433 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 23:02:03,663 [I] connection success
2026-08-08 23:02:03,663 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 23:02:03,663 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 23:02:03,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 23:02:03,889 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 23:02:03,889 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 23:02:03,889 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 23:02:03,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.194.20:41121/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 23:02:04,132 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 23:02:04.567899 24328 mini_ranger_kms.cc:310] Created kms service
I20260808 23:02:04.760537 24328 mini_ranger_kms.cc:326] Created kudu user
I20260808 23:02:04.801752 24328 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 23:02:04 dist-test-slave-f0tn krb5kdc[10278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230122, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.23.194.20@KRBTEST.COM
I20260808 23:02:05.166306 24328 mini_ranger_kms.cc:384] Added ranger policy
I20260808 23:02:05.166591 24328 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ranger-kms:/tmp/dist-test-taski6UDxc/build/release/bin/postgresql.jar:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taski6UDxc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 23:02:05.166622 24328 mini_ranger_kms.cc:240] Using host: 127.23.194.20
I20260808 23:02:05.171139 24328 mini_ranger_kms.cc:276] Ranger KMS PID: 10891
I20260808 23:02:05.171205 24328 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.23.194.20:37123
Aug 08, 2026 11:02:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 11:02:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 11:02:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 11:02:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 11:02:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 11:02:06 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 23:02:07 dist-test-slave-f0tn krb5kdc[10278](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786230127, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.194.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:07.659766 24328 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.125s user 0.006s sys 0.136s
I20260808 23:02:07.659857 24328 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 23:02:07.659895 24328 mini_ranger_kms.cc:401] 127.23.194.20:37123/kms/v1/keys
Aug 08 23:02:08 dist-test-slave-f0tn krb5kdc[10278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230127, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.194.20@KRBTEST.COM for HTTP/127.23.194.20@KRBTEST.COM
Aug 08 23:02:08 dist-test-slave-f0tn krb5kdc[10278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230127, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.194.20@KRBTEST.COM for HTTP/127.23.194.20@KRBTEST.COM
Aug 08 23:02:10 dist-test-slave-f0tn krb5kdc[10278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:02:10.631587 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:36701
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:36701
--ranger_config_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.194.20:37123/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:02:10.741858 10979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:10.742077 10979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:10.742108 10979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:10.743273 10979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:02:10.743314 10979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:10.743332 10979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:10.743350 10979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:02:10.743364 10979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:02:10.744954 10979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/wal
--ranger_kms_url=127.23.194.20:37123/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:36701
--ranger_config_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:36701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:10.745215 10979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:10.745468 10979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:10.750569 10986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:10.750546 10985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:10.750553 10989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:10.750739 10979 server_base.cc:1034] running on GCE node
I20260808 23:02:10.751084 10979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:10.751372 10979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:10.752519 10979 hybrid_clock.cc:648] HybridClock initialized: now 1786230130752496 us; error 39 us; skew 500 ppm
Aug 08 23:02:10 dist-test-slave-f0tn krb5kdc[10278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230130, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:10.757758 10979 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:02:10.758236 10979 webserver.cc:460] Webserver started at http://127.23.194.62:34623/ using document root <none> and password file <none>
I20260808 23:02:10.758471 10979 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:10.758514 10979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:10.758654 10979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:02:10 dist-test-slave-f0tn krb5kdc[10278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230130, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for HTTP/127.23.194.20@KRBTEST.COM
I20260808 23:02:10.804131 10979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "d71fdef57f394e6c850581f17e5ef83a"
format_stamp: "Formatted at 2026-08-08 23:02:10 on dist-test-slave-f0tn"
server_key: "1ca89aadbf307ddb71c20ed21e13c5e2"
server_key_iv: "8e43d0a3a6c2d76a0f7aa40102e8c2ee"
server_key_version: "kuduclusterkey@0"
I20260808 23:02:10.804594 10979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "d71fdef57f394e6c850581f17e5ef83a"
format_stamp: "Formatted at 2026-08-08 23:02:10 on dist-test-slave-f0tn"
server_key: "1ca89aadbf307ddb71c20ed21e13c5e2"
server_key_iv: "8e43d0a3a6c2d76a0f7aa40102e8c2ee"
server_key_version: "kuduclusterkey@0"
I20260808 23:02:10.806468 10979 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 23:02:10.819635 11001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:10.819896 10979 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:02:10.820051 10979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "d71fdef57f394e6c850581f17e5ef83a"
format_stamp: "Formatted at 2026-08-08 23:02:10 on dist-test-slave-f0tn"
server_key: "1ca89aadbf307ddb71c20ed21e13c5e2"
server_key_iv: "8e43d0a3a6c2d76a0f7aa40102e8c2ee"
server_key_version: "kuduclusterkey@0"
I20260808 23:02:10.820195 10979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:10.833371 10979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:10.835240 10979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:10.835394 10979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:10.839846 10979 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:36701
I20260808 23:02:10.839882 11078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:36701 every 8 connection(s)
I20260808 23:02:10.840248 10979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:02:10.841040 11079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:02:10.844308 11079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a: Bootstrap starting.
I20260808 23:02:10.845067 11079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a: Neither blocks nor log segments found. Creating new log.
I20260808 23:02:10.845404 11079 log.cc:824] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:10.846165 11079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a: No bootstrap required, opened a new log
I20260808 23:02:10.848317 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 10979
I20260808 23:02:10.848420 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:02:10.848578 24328 external_mini_cluster.cc:1428] Setting key 3682b087951a57f15be824f83439efc8
I20260808 23:02:10.848904 11079 raft_consensus.cc:348] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71fdef57f394e6c850581f17e5ef83a" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36701 } }
I20260808 23:02:10.849211 11079 raft_consensus.cc:374] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:02:10.849323 11079 raft_consensus.cc:729] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d71fdef57f394e6c850581f17e5ef83a, State: Initialized, Role: FOLLOWER
I20260808 23:02:10.849506 11079 consensus_queue.cc:260] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71fdef57f394e6c850581f17e5ef83a" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36701 } }
I20260808 23:02:10.849591 11079 raft_consensus.cc:388] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:02:10.849620 11079 raft_consensus.cc:482] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:02:10.849685 11079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:02:10.850392 11079 raft_consensus.cc:504] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71fdef57f394e6c850581f17e5ef83a" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36701 } }
I20260808 23:02:10.850502 11079 leader_election.cc:302] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d71fdef57f394e6c850581f17e5ef83a; no voters:
I20260808 23:02:10.850653 11079 leader_election.cc:288] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:02:10.850689 11090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:02:10.850912 11090 raft_consensus.cc:686] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [term 1 LEADER]: Becoming Leader. State: Replica: d71fdef57f394e6c850581f17e5ef83a, State: Running, Role: LEADER
I20260808 23:02:10.850980 11079 sys_catalog.cc:564] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:02:10.851046 11090 consensus_queue.cc:237] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71fdef57f394e6c850581f17e5ef83a" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36701 } }
I20260808 23:02:10.851431 11090 sys_catalog.cc:455] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d71fdef57f394e6c850581f17e5ef83a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71fdef57f394e6c850581f17e5ef83a" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36701 } } }
I20260808 23:02:10.851549 11090 sys_catalog.cc:458] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [sys.catalog]: This master's current role is: LEADER
I20260808 23:02:10.851708 11090 sys_catalog.cc:455] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d71fdef57f394e6c850581f17e5ef83a. Latest consensus state: current_term: 1 leader_uuid: "d71fdef57f394e6c850581f17e5ef83a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71fdef57f394e6c850581f17e5ef83a" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36701 } } }
I20260808 23:02:10.851811 11090 sys_catalog.cc:458] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [sys.catalog]: This master's current role is: LEADER
I20260808 23:02:10.851974 11107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:02:10.852520 11107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:02:10.852772 11079 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 08 23:02:10 dist-test-slave-f0tn krb5kdc[10278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:10.862893 11089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:10.849079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40362 (local address 127.23.194.62:36701)
0808 23:02:10.849275 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:10.849284 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:10.849317 (+ 33us) server_negotiation.cc:407] Connection header received
0808 23:02:10.849439 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:10.849447 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:10.849504 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:10.849631 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:10.850328 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:10.851717 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:10.852895 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:10.855906 (+ 3011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:10.856399 (+ 493us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:10.858853 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:10.858858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:10.858859 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:10.858871 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:10.860762 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:10.861165 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:10.861168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:10.861169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:10.861277 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:10.861494 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:10.861495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:10.861495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:10.861809 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:10.862376 (+ 567us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:10.862448 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 23:02:10.862639 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":89,"threads_started":1}
I20260808 23:02:10.864028 11107 catalog_manager.cc:1347] Generated new cluster ID: 480e69ca32954e63b9116d07b3e7479e
I20260808 23:02:10.864081 11107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:02:10.881634 11107 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:02:10.882423 11107 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:02:10.886430 11107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a: Generated new TSK 0
I20260808 23:02:10.886669 11107 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
I20260808 23:02:11.589365 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.194.20:37123/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:36701
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:02:11.676301 11158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:11.676506 11158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:11.676537 11158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:11.677748 11158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:11.677790 11158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:11.677825 11158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:02:11.679348 11158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.23.194.20:37123/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:36701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:11.679600 11158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:11.679849 11158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:11.685190 11167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:11.685359 11169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:11.685587 11158 server_base.cc:1034] running on GCE node
W20260808 23:02:11.685209 11166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:11.686074 11158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:11.686362 11158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:11.687542 11158 hybrid_clock.cc:648] HybridClock initialized: now 1786230131687513 us; error 45 us; skew 500 ppm
Aug 08 23:02:11 dist-test-slave-f0tn krb5kdc[10278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230131, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:11.690980 11158 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 23:02:11.691588 11158 webserver.cc:460] Webserver started at http://127.23.194.1:38569/ using document root <none> and password file <none>
I20260808 23:02:11.691910 11158 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:11.691989 11158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:11.692154 11158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:02:11 dist-test-slave-f0tn krb5kdc[10278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230131, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for HTTP/127.23.194.20@KRBTEST.COM
I20260808 23:02:11.707239 11158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "5dcdb9da52b1488c96297576ccab4cc3"
format_stamp: "Formatted at 2026-08-08 23:02:11 on dist-test-slave-f0tn"
server_key: "715e378314df7550f8e8f79240c647e0"
server_key_iv: "690ed4d1dabe38a4a2777650e0296848"
server_key_version: "kuduclusterkey@0"
I20260808 23:02:11.707764 11158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "5dcdb9da52b1488c96297576ccab4cc3"
format_stamp: "Formatted at 2026-08-08 23:02:11 on dist-test-slave-f0tn"
server_key: "715e378314df7550f8e8f79240c647e0"
server_key_iv: "690ed4d1dabe38a4a2777650e0296848"
server_key_version: "kuduclusterkey@0"
I20260808 23:02:11.709715 11158 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 23:02:11.730530 11177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:11.730798 11158 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260808 23:02:11.730930 11158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "5dcdb9da52b1488c96297576ccab4cc3"
format_stamp: "Formatted at 2026-08-08 23:02:11 on dist-test-slave-f0tn"
server_key: "715e378314df7550f8e8f79240c647e0"
server_key_iv: "690ed4d1dabe38a4a2777650e0296848"
server_key_version: "kuduclusterkey@0"
I20260808 23:02:11.731048 11158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:11.750144 11158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:11.752426 11158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:11.752589 11158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:11.752841 11158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:02:11.753240 11158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:02:11.753290 11158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:11.753328 11158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:02:11.753352 11158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:11.760774 11158 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:44637
I20260808 23:02:11.760846 11373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:44637 every 8 connection(s)
I20260808 23:02:11.761233 11158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:02:11.765628 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 11158
I20260808 23:02:11.765723 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:02:11.765930 24328 external_mini_cluster.cc:1428] Setting key 5b741da93ef55f7ad2c2ddb86aec6dca
Aug 08 23:02:11 dist-test-slave-f0tn krb5kdc[10278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230131, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:11.772755 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:11.762063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:52857 (local address 127.23.194.62:36701)
0808 23:02:11.762229 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:11.762232 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:11.762300 (+ 68us) server_negotiation.cc:407] Connection header received
0808 23:02:11.762987 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:11.762989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:11.763004 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:11.763097 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:11.763877 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:11.764687 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:11.766374 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:11.766713 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:11.766776 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:11.769356 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:11.769358 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:11.769358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:11.769365 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:11.771160 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:11.771637 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:11.771638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:11.771639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:11.771711 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:11.772004 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:11.772005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:11.772005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:11.772148 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:11.772225 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:11.772565 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 23:02:11.772653 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":100,"threads_started":1}
I20260808 23:02:11.772888 11379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:11.762007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:36701 (local address 127.23.194.1:52857)
0808 23:02:11.762179 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:11.762221 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:11.762846 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:11.763196 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:11.763201 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:11.763455 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:11.763753 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:11.763760 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:11.764810 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:11.764812 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:11.766288 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:11.766291 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:11.766857 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:11.766859 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:11.767010 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:11.767640 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:11.767650 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:11.769247 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:11.771252 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:11.771259 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:11.771261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:11.771547 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:11.771786 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:11.771787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:11.771787 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:11.771924 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:11.772226 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:11.772232 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:11.772387 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 23:02:11.772499 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 23:02:11.772661 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260808 23:02:11.773465 11375 heartbeater.cc:347] Connected to a master server at 127.23.194.62:36701
I20260808 23:02:11.773581 11375 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:11.773780 11375 heartbeater.cc:511] Master 127.23.194.62:36701 requested a full tablet report, sending...
I20260808 23:02:11.774348 11026 ts_manager.cc:194] Registered new tserver with Master: 5dcdb9da52b1488c96297576ccab4cc3 (127.23.194.1:44637)
I20260808 23:02:11.775234 11026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:52857
WARNING: no policy specified for kudu/127.23.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
I20260808 23:02:11.797971 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.194.20:37123/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:36701
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:02:11 dist-test-slave-f0tn krb5kdc[10278](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786230131, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 23:02:11.919617 11386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:11.919955 11386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:11.920042 11386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:11.922103 11386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:11.922284 11386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:11.922429 11386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:02:11.925472 11386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.23.194.20:37123/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:36701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:11.925942 11386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:11.926270 11386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:11.932689 11398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:11.932826 11403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:11.932689 11401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:11.934019 11386 server_base.cc:1034] running on GCE node
I20260808 23:02:11.934274 11386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:11.934540 11386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:11.937808 11386 hybrid_clock.cc:648] HybridClock initialized: now 1786230131937768 us; error 48 us; skew 500 ppm
Aug 08 23:02:11 dist-test-slave-f0tn krb5kdc[10278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230131, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:11.941008 11386 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 23:02:11.941568 11386 webserver.cc:460] Webserver started at http://127.23.194.2:34243/ using document root <none> and password file <none>
I20260808 23:02:11.941866 11386 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:11.941932 11386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:11.942063 11386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:02:11 dist-test-slave-f0tn krb5kdc[10278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230131, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for HTTP/127.23.194.20@KRBTEST.COM
I20260808 23:02:11.955215 11386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "c67d584ecea04738bef997b6b44c2a41"
format_stamp: "Formatted at 2026-08-08 23:02:11 on dist-test-slave-f0tn"
server_key: "3872be92190c8c17ee1fb92a0e670ef7"
server_key_iv: "a5f7e71ff8cdcc218730dba8ee2fd42b"
server_key_version: "kuduclusterkey@0"
I20260808 23:02:11.955755 11386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "c67d584ecea04738bef997b6b44c2a41"
format_stamp: "Formatted at 2026-08-08 23:02:11 on dist-test-slave-f0tn"
server_key: "3872be92190c8c17ee1fb92a0e670ef7"
server_key_iv: "a5f7e71ff8cdcc218730dba8ee2fd42b"
server_key_version: "kuduclusterkey@0"
I20260808 23:02:11.958577 11386 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260808 23:02:11.975032 11413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:11.975296 11386 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260808 23:02:11.975420 11386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "c67d584ecea04738bef997b6b44c2a41"
format_stamp: "Formatted at 2026-08-08 23:02:11 on dist-test-slave-f0tn"
server_key: "3872be92190c8c17ee1fb92a0e670ef7"
server_key_iv: "a5f7e71ff8cdcc218730dba8ee2fd42b"
server_key_version: "kuduclusterkey@0"
I20260808 23:02:11.975533 11386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:11.995070 11386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:11.997592 11386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:11.997756 11386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:11.998021 11386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:02:11.998427 11386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:02:11.998485 11386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:11.998569 11386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:02:11.998620 11386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:12.004910 11386 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:36091
I20260808 23:02:12.005332 11386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:02:12.007294 11598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:36091 every 8 connection(s)
I20260808 23:02:12.014366 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 11386
I20260808 23:02:12.014482 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:02:12.014675 24328 external_mini_cluster.cc:1428] Setting key 125894b83326a63dc4359300244d24dd
Aug 08 23:02:12 dist-test-slave-f0tn krb5kdc[10278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230131, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:12.018980 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:12.007741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:56909 (local address 127.23.194.62:36701)
0808 23:02:12.007793 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:12.007795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:12.008057 (+ 262us) server_negotiation.cc:407] Connection header received
0808 23:02:12.008729 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:12.008731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:12.008748 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:12.008809 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:12.009860 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:12.010639 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:12.012821 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:12.013150 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:12.013236 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:12.015776 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:12.015778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:12.015778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:12.015786 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:12.017448 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:12.017907 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:12.017908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:12.017909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:12.017970 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:12.018241 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:12.018241 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:12.018242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:12.018364 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:12.018405 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:12.018770 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 23:02:12.018858 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 23:02:12.019686 11606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:12.007698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:36701 (local address 127.23.194.2:56909)
0808 23:02:12.007876 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:12.007929 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:12.008596 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:12.008893 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:12.008897 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:12.009149 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:12.009456 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:12.009463 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:12.010756 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:12.010758 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:12.012712 (+ 1954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:12.012716 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:12.013296 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:12.013298 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:12.013478 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:12.014056 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:12.014067 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:12.015660 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:12.017536 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:12.017543 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:12.017546 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:12.017822 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:12.018035 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:12.018036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:12.018036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:12.018172 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:12.018452 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:12.018459 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:12.018620 (+ 161us) client_negotiation.cc:769] Sending connection context
0808 23:02:12.019309 (+ 689us) client_negotiation.cc:240] Negotiation successful
0808 23:02:12.019466 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260808 23:02:12.020462 11603 heartbeater.cc:347] Connected to a master server at 127.23.194.62:36701
I20260808 23:02:12.020566 11603 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:12.020776 11603 heartbeater.cc:511] Master 127.23.194.62:36701 requested a full tablet report, sending...
I20260808 23:02:12.021153 11026 ts_manager.cc:194] Registered new tserver with Master: c67d584ecea04738bef997b6b44c2a41 (127.23.194.2:36091)
I20260808 23:02:12.021664 11026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:56909
WARNING: no policy specified for kudu/127.23.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
I20260808 23:02:12.046378 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.194.20:37123/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:36701
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:02:12.171654 11611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:12.171854 11611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:12.171882 11611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:12.173035 11611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:12.173075 11611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:12.173110 11611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:02:12.174584 11611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.23.194.20:37123/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:36701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:12.174815 11611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:12.175026 11611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:12.180769 11617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:12.180809 11616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:12.180823 11620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:12.181231 11611 server_base.cc:1034] running on GCE node
I20260808 23:02:12.181463 11611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:12.181684 11611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:12.184907 11611 hybrid_clock.cc:648] HybridClock initialized: now 1786230132184892 us; error 25 us; skew 500 ppm
Aug 08 23:02:12 dist-test-slave-f0tn krb5kdc[10278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230132, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:12.187557 11611 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 23:02:12.188127 11611 webserver.cc:460] Webserver started at http://127.23.194.3:45971/ using document root <none> and password file <none>
I20260808 23:02:12.188442 11611 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:12.188516 11611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:12.188668 11611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:02:12 dist-test-slave-f0tn krb5kdc[10278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230132, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for HTTP/127.23.194.20@KRBTEST.COM
I20260808 23:02:12.207382 11611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "f35dcf39324944d495aed69610537c95"
format_stamp: "Formatted at 2026-08-08 23:02:12 on dist-test-slave-f0tn"
server_key: "96218671eb89946f2dcd2bfa0b966c60"
server_key_iv: "ea4cf8b8912f2580cd0fc0ab5ed9039b"
server_key_version: "kuduclusterkey@0"
I20260808 23:02:12.207845 11611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "f35dcf39324944d495aed69610537c95"
format_stamp: "Formatted at 2026-08-08 23:02:12 on dist-test-slave-f0tn"
server_key: "96218671eb89946f2dcd2bfa0b966c60"
server_key_iv: "ea4cf8b8912f2580cd0fc0ab5ed9039b"
server_key_version: "kuduclusterkey@0"
I20260808 23:02:12.213714 11611 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260808 23:02:12.227524 11632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:12.227773 11611 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:02:12.227885 11611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "f35dcf39324944d495aed69610537c95"
format_stamp: "Formatted at 2026-08-08 23:02:12 on dist-test-slave-f0tn"
server_key: "96218671eb89946f2dcd2bfa0b966c60"
server_key_iv: "ea4cf8b8912f2580cd0fc0ab5ed9039b"
server_key_version: "kuduclusterkey@0"
I20260808 23:02:12.227983 11611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:12.248323 11611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:12.250681 11611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:12.250844 11611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:12.251058 11611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:02:12.251466 11611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:02:12.251510 11611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:12.251543 11611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:02:12.251564 11611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:12.260455 11611 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:33713
I20260808 23:02:12.260912 11611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 23:02:12.263957 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 11611
I20260808 23:02:12.264065 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:02:12.264247 24328 external_mini_cluster.cc:1428] Setting key bc0bac5bc1a3be4507e701d021bc464a
I20260808 23:02:12.265960 11806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:33713 every 8 connection(s)
Aug 08 23:02:12 dist-test-slave-f0tn krb5kdc[10278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230132, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:12.289592 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:12.273102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:60379 (local address 127.23.194.62:36701)
0808 23:02:12.273152 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:12.273154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:12.273565 (+ 411us) server_negotiation.cc:407] Connection header received
0808 23:02:12.274272 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:12.274273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:12.274286 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:12.274312 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:12.275774 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:12.276432 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:12.277858 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:12.278225 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:12.278270 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:12.284624 (+ 6354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:12.284626 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:12.284626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:12.284635 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:12.288003 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:12.288506 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:12.288507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:12.288508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:12.288568 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:12.288875 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:12.288876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:12.288876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:12.289013 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:12.289071 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:12.289414 (+ 343us) server_negotiation.cc:299] Negotiation successful
0808 23:02:12.289499 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 23:02:12.289820 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:12.272663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:36701 (local address 127.23.194.3:60379)
0808 23:02:12.273432 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:12.273484 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:12.274157 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:12.274508 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:12.274512 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:12.275102 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:12.275678 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:12.275686 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:12.276540 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:12.276541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:12.277763 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:12.277767 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:12.278406 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:12.278407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:12.278559 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:12.279182 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:12.279193 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:12.284343 (+ 5150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:12.288091 (+ 3748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:12.288098 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:12.288102 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:12.288408 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:12.288661 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:12.288661 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:12.288662 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:12.288799 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:12.289104 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:12.289110 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:12.289247 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 23:02:12.289333 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 23:02:12.289511 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":225,"threads_started":1}
I20260808 23:02:12.290427 11807 heartbeater.cc:347] Connected to a master server at 127.23.194.62:36701
I20260808 23:02:12.290526 11807 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:12.290717 11807 heartbeater.cc:511] Master 127.23.194.62:36701 requested a full tablet report, sending...
I20260808 23:02:12.291071 11026 ts_manager.cc:194] Registered new tserver with Master: f35dcf39324944d495aed69610537c95 (127.23.194.3:33713)
I20260808 23:02:12.291426 11026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:60379
I20260808 23:02:12.297211 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:02:12.298076 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 11158
I20260808 23:02:12.314584 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 11386
I20260808 23:02:12.323606 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 11611
I20260808 23:02:12.343518 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 10979
I20260808 23:02:12.355325 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:36701
--webserver_interface=127.23.194.62
--webserver_port=34623
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:36701
--ranger_config_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.194.20:37123/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.194.20:37123/kms with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:02:12.438639 11811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:12.438845 11811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:12.438867 11811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:12.440025 11811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:02:12.440069 11811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:12.440085 11811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:12.440097 11811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:02:12.440114 11811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:02:12.441851 11811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.194.20:37123/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:36701
--ranger_config_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:36701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=34623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:12.442055 11811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:12.442234 11811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:12.447568 11823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:12.447610 11811 server_base.cc:1034] running on GCE node
W20260808 23:02:12.447795 11824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:12.447654 11827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:12.448125 11811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:12.448407 11811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:12.449548 11811 hybrid_clock.cc:648] HybridClock initialized: now 1786230132449535 us; error 31 us; skew 500 ppm
Aug 08 23:02:12 dist-test-slave-f0tn krb5kdc[10278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230132, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:12.456872 11811 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:02:12.457442 11811 webserver.cc:460] Webserver started at http://127.23.194.62:34623/ using document root <none> and password file <none>
I20260808 23:02:12.457665 11811 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:12.457726 11811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 23:02:12 dist-test-slave-f0tn krb5kdc[10278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230132, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for HTTP/127.23.194.20@KRBTEST.COM
I20260808 23:02:12.495994 11811 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:02:12.496752 11839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:12.496915 11811 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:02:12.497013 11811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "d71fdef57f394e6c850581f17e5ef83a"
format_stamp: "Formatted at 2026-08-08 23:02:10 on dist-test-slave-f0tn"
server_key: "1ca89aadbf307ddb71c20ed21e13c5e2"
server_key_iv: "8e43d0a3a6c2d76a0f7aa40102e8c2ee"
server_key_version: "kuduclusterkey@0"
I20260808 23:02:12.497363 11811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:12.513021 11811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:12.515095 11811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:12.515247 11811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:12.520440 11811 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:36701
I20260808 23:02:12.520577 11919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:36701 every 8 connection(s)
I20260808 23:02:12.520877 11811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:02:12.521029 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 11811
I20260808 23:02:12.521333 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.194.20:37123/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:44637
--local_ip_for_outbound_sockets=127.23.194.1
--tserver_master_addrs=127.23.194.62:36701
--webserver_port=38569
--webserver_interface=127.23.194.1
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.194.20:37123/kms with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:02:12.522599 11920 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:02:12.523402 11920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a: Bootstrap starting.
I20260808 23:02:12.533617 11920 log.cc:824] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:12.535651 11920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:02:12.535938 11920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a: Bootstrap complete.
I20260808 23:02:12.539057 11920 raft_consensus.cc:348] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71fdef57f394e6c850581f17e5ef83a" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36701 } }
I20260808 23:02:12.539372 11920 raft_consensus.cc:729] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d71fdef57f394e6c850581f17e5ef83a, State: Initialized, Role: FOLLOWER
I20260808 23:02:12.539530 11920 consensus_queue.cc:260] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71fdef57f394e6c850581f17e5ef83a" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36701 } }
I20260808 23:02:12.539590 11920 raft_consensus.cc:388] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:02:12.539633 11920 raft_consensus.cc:482] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:02:12.539676 11920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [term 1 FOLLOWER]: Advancing to term 2
I20260808 23:02:12.540889 11920 raft_consensus.cc:504] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71fdef57f394e6c850581f17e5ef83a" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36701 } }
I20260808 23:02:12.540994 11920 leader_election.cc:302] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d71fdef57f394e6c850581f17e5ef83a; no voters:
I20260808 23:02:12.541164 11920 leader_election.cc:288] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 23:02:12.541208 11927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [term 2 FOLLOWER]: Leader election won for term 2
I20260808 23:02:12.541350 11927 raft_consensus.cc:686] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [term 2 LEADER]: Becoming Leader. State: Replica: d71fdef57f394e6c850581f17e5ef83a, State: Running, Role: LEADER
I20260808 23:02:12.541414 11920 sys_catalog.cc:564] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:02:12.541466 11927 consensus_queue.cc:237] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71fdef57f394e6c850581f17e5ef83a" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36701 } }
I20260808 23:02:12.541745 11927 sys_catalog.cc:455] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d71fdef57f394e6c850581f17e5ef83a. Latest consensus state: current_term: 2 leader_uuid: "d71fdef57f394e6c850581f17e5ef83a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71fdef57f394e6c850581f17e5ef83a" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36701 } } }
I20260808 23:02:12.541821 11927 sys_catalog.cc:458] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [sys.catalog]: This master's current role is: LEADER
I20260808 23:02:12.541967 11930 sys_catalog.cc:455] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d71fdef57f394e6c850581f17e5ef83a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71fdef57f394e6c850581f17e5ef83a" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36701 } } }
I20260808 23:02:12.542025 11930 sys_catalog.cc:458] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a [sys.catalog]: This master's current role is: LEADER
I20260808 23:02:12.542455 11934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:02:12.542871 11934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:02:12.542984 11934 catalog_manager.cc:1259] Loaded cluster ID: 480e69ca32954e63b9116d07b3e7479e
I20260808 23:02:12.543020 11934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:02:12.543198 11920 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 23:02:12.546267 11934 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:02:12.546763 11934 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a: Loaded TSK: 0
I20260808 23:02:12.547005 11934 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 23:02:12.647228 11923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:12.647648 11923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:12.647770 11923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:12.649855 11923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:12.649966 11923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:12.650060 11923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:02:12.652503 11923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.194.20:37123/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:44637
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=38569
--tserver_master_addrs=127.23.194.62:36701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:12.652989 11923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:12.653330 11923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:12.661950 11976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:12.662909 11923 server_base.cc:1034] running on GCE node
W20260808 23:02:12.662956 11978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:12.663193 11975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:12.663475 11923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:12.663805 11923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:12.667137 11923 hybrid_clock.cc:648] HybridClock initialized: now 1786230132667095 us; error 52 us; skew 500 ppm
Aug 08 23:02:12 dist-test-slave-f0tn krb5kdc[10278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230132, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:12.671473 11923 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 23:02:12.672087 11923 webserver.cc:460] Webserver started at http://127.23.194.1:38569/ using document root <none> and password file <none>
I20260808 23:02:12.672422 11923 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:12.672513 11923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 23:02:12 dist-test-slave-f0tn krb5kdc[10278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230132, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for HTTP/127.23.194.20@KRBTEST.COM
I20260808 23:02:12.705911 11923 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.000s sys 0.001s
I20260808 23:02:12.707302 11989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:12.707525 11923 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 23:02:12.707636 11923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "5dcdb9da52b1488c96297576ccab4cc3"
format_stamp: "Formatted at 2026-08-08 23:02:11 on dist-test-slave-f0tn"
server_key: "715e378314df7550f8e8f79240c647e0"
server_key_iv: "690ed4d1dabe38a4a2777650e0296848"
server_key_version: "kuduclusterkey@0"
I20260808 23:02:12.708103 11923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:12.721711 11923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:12.723728 11923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:12.723884 11923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:12.724128 11923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:02:12.724500 11923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:02:12.724563 11923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:12.724601 11923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:02:12.724623 11923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:12.730849 11923 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:44637
I20260808 23:02:12.731340 11923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:02:12.734133 12186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:44637 every 8 connection(s)
I20260808 23:02:12.739619 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 11923
I20260808 23:02:12.739944 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.194.20:37123/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:36091
--local_ip_for_outbound_sockets=127.23.194.2
--tserver_master_addrs=127.23.194.62:36701
--webserver_port=34243
--webserver_interface=127.23.194.2
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.194.20:37123/kms with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:02:12 dist-test-slave-f0tn krb5kdc[10278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230132, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:12.763367 12197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:12.735155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:44987 (local address 127.23.194.62:36701)
0808 23:02:12.736999 (+ 1844us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:12.737007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:12.737033 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:02:12.737151 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:12.737155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:12.737208 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:12.737333 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:12.737965 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:12.738926 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:12.739917 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:12.740394 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:12.740503 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:12.749896 (+ 9393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:12.749901 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:12.749902 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:12.749916 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:12.754978 (+ 5062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:12.761613 (+ 6635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:12.761618 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:12.761618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:12.761782 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:12.762119 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:12.762120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:12.762121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:12.762463 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:12.762539 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:12.762973 (+ 434us) server_negotiation.cc:299] Negotiation successful
0808 23:02:12.763131 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1793,"thread_start_us":85,"threads_started":1}
I20260808 23:02:12.763989 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:12.734789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:36701 (local address 127.23.194.1:44987)
0808 23:02:12.736261 (+ 1472us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:12.736309 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:12.736798 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:12.737419 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:12.737423 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:12.737647 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:12.737856 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:12.737860 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:12.739024 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:12.739026 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:12.739839 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:12.739842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:12.740499 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:12.740500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:12.740633 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:12.741137 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:12.741147 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:12.749736 (+ 8589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:12.761020 (+ 11284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:12.761032 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:12.761035 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:12.761492 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:12.761871 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:12.761872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:12.761873 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:12.762028 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:12.762579 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:12.762585 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:12.762766 (+ 181us) client_negotiation.cc:769] Sending connection context
0808 23:02:12.763576 (+ 810us) client_negotiation.cc:240] Negotiation successful
0808 23:02:12.763754 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1421,"thread_start_us":84,"threads_started":1}
I20260808 23:02:12.764943 12188 heartbeater.cc:347] Connected to a master server at 127.23.194.62:36701
I20260808 23:02:12.765070 12188 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:12.765276 12188 heartbeater.cc:511] Master 127.23.194.62:36701 requested a full tablet report, sending...
W20260808 23:02:12.766315 12188 heartbeater.cc:644] Failed to heartbeat to 127.23.194.62:36701 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 23:02:12.766393 12188 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:12.766462 12188 heartbeater.cc:511] Master 127.23.194.62:36701 requested a full tablet report, sending...
I20260808 23:02:12.766907 12188 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:12.766985 12188 heartbeater.cc:511] Master 127.23.194.62:36701 requested a full tablet report, sending...
W20260808 23:02:12.767349 12188 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 23:02:12.847615 12199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:12.847891 12199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:12.847929 12199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:12.849611 12199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:12.849673 12199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:12.849716 12199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:02:12.852046 12199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.194.20:37123/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:36091
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=34243
--tserver_master_addrs=127.23.194.62:36701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:12.852294 12199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:12.852547 12199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:12.859004 12208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:12.859017 12205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:12.859290 12199 server_base.cc:1034] running on GCE node
W20260808 23:02:12.859018 12210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:12.859717 12199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:12.859997 12199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:12.861153 12199 hybrid_clock.cc:648] HybridClock initialized: now 1786230132861148 us; error 31 us; skew 500 ppm
Aug 08 23:02:12 dist-test-slave-f0tn krb5kdc[10278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230132, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:12.863813 12199 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 23:02:12.864266 12199 webserver.cc:460] Webserver started at http://127.23.194.2:34243/ using document root <none> and password file <none>
I20260808 23:02:12.864480 12199 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:12.864542 12199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 23:02:12 dist-test-slave-f0tn krb5kdc[10278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230132, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for HTTP/127.23.194.20@KRBTEST.COM
I20260808 23:02:12.884318 12199 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.001s
I20260808 23:02:12.885324 12226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:12.885488 12199 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 23:02:12.885577 12199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "c67d584ecea04738bef997b6b44c2a41"
format_stamp: "Formatted at 2026-08-08 23:02:11 on dist-test-slave-f0tn"
server_key: "3872be92190c8c17ee1fb92a0e670ef7"
server_key_iv: "a5f7e71ff8cdcc218730dba8ee2fd42b"
server_key_version: "kuduclusterkey@0"
I20260808 23:02:12.885946 12199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:12.899628 12199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:12.901907 12199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:12.902081 12199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:12.902294 12199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:02:12.902657 12199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:02:12.902707 12199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:12.902737 12199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:02:12.902761 12199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:12.910557 12199 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:36091
I20260808 23:02:12.910982 12199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:02:12.911473 12402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:36091 every 8 connection(s)
I20260808 23:02:12.911681 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 12199
I20260808 23:02:12.911988 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.194.20:37123/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:33713
--local_ip_for_outbound_sockets=127.23.194.3
--tserver_master_addrs=127.23.194.62:36701
--webserver_port=45971
--webserver_interface=127.23.194.3
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.194.20:37123/kms with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:02:12 dist-test-slave-f0tn krb5kdc[10278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230132, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:12.922024 12197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:12.911940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:34919 (local address 127.23.194.62:36701)
0808 23:02:12.911987 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:12.911989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:12.912404 (+ 415us) server_negotiation.cc:407] Connection header received
0808 23:02:12.912896 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:12.912898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:12.912916 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:12.912968 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:12.913564 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:12.914350 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:12.915803 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:12.916107 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:12.916197 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:12.918724 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:12.918726 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:12.918726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:12.918735 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:12.920376 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:12.920869 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:12.920870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:12.920871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:12.920937 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:12.921239 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:12.921240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:12.921241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:12.921407 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:12.921479 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:12.921837 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 23:02:12.921928 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 23:02:12.922588 12406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:12.911942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:36701 (local address 127.23.194.2:34919)
0808 23:02:12.912273 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:12.912308 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:12.912784 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:12.913044 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:12.913049 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:12.913215 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:12.913436 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:12.913443 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:12.914471 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:12.914472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:12.915672 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:12.915676 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:12.916221 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:12.916222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:12.916354 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:12.916941 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:12.916950 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:12.918604 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:12.920469 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:12.920476 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:12.920479 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:12.920775 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:12.921015 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:12.921016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:12.921017 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:12.921157 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:12.921470 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:12.921477 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:12.921652 (+ 175us) client_negotiation.cc:769] Sending connection context
0808 23:02:12.921792 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 23:02:12.921951 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":86,"threads_started":1}
I20260808 23:02:12.923192 12404 heartbeater.cc:347] Connected to a master server at 127.23.194.62:36701
I20260808 23:02:12.923286 12404 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:12.923475 12404 heartbeater.cc:511] Master 127.23.194.62:36701 requested a full tablet report, sending...
W20260808 23:02:12.924204 12404 heartbeater.cc:644] Failed to heartbeat to 127.23.194.62:36701 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 23:02:12.924274 12404 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:12.924333 12404 heartbeater.cc:511] Master 127.23.194.62:36701 requested a full tablet report, sending...
I20260808 23:02:12.924702 12404 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:12.924777 12404 heartbeater.cc:511] Master 127.23.194.62:36701 requested a full tablet report, sending...
W20260808 23:02:12.925108 12404 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 23:02:13.013830 12410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:13.014070 12410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:13.014103 12410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:13.015270 12410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:13.015343 12410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:13.015379 12410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:02:13.017560 12410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.194.20:37123/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:33713
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=45971
--tserver_master_addrs=127.23.194.62:36701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:13.017808 12410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:13.018088 12410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:13.023365 12420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:13.023365 12419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:13.023732 12422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:13.023774 12410 server_base.cc:1034] running on GCE node
I20260808 23:02:13.024070 12410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:13.024323 12410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:13.025444 12410 hybrid_clock.cc:648] HybridClock initialized: now 1786230133025421 us; error 33 us; skew 500 ppm
Aug 08 23:02:13 dist-test-slave-f0tn krb5kdc[10278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230133, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:13.027921 12410 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 23:02:13.028362 12410 webserver.cc:460] Webserver started at http://127.23.194.3:45971/ using document root <none> and password file <none>
I20260808 23:02:13.028604 12410 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:13.028667 12410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 23:02:13 dist-test-slave-f0tn krb5kdc[10278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230133, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for HTTP/127.23.194.20@KRBTEST.COM
I20260808 23:02:13.048309 12410 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:02:13.053678 12436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:13.053912 12410 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.000s
I20260808 23:02:13.054050 12410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "f35dcf39324944d495aed69610537c95"
format_stamp: "Formatted at 2026-08-08 23:02:12 on dist-test-slave-f0tn"
server_key: "96218671eb89946f2dcd2bfa0b966c60"
server_key_iv: "ea4cf8b8912f2580cd0fc0ab5ed9039b"
server_key_version: "kuduclusterkey@0"
I20260808 23:02:13.054559 12410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:13.082762 12410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:13.084720 12410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:13.084864 12410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:13.085099 12410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:02:13.085457 12410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:02:13.085507 12410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:13.085536 12410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:02:13.085558 12410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:13.092746 12410 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:33713
I20260808 23:02:13.092784 12621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:33713 every 8 connection(s)
I20260808 23:02:13.093240 12410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
Aug 08 23:02:13 dist-test-slave-f0tn krb5kdc[10278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230133, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:13.101191 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 12410
I20260808 23:02:13.104032 12624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:13.093984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:36701 (local address 127.23.194.3:40143)
0808 23:02:13.094162 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:13.094212 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:13.094845 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:13.095232 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:13.095236 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:13.095460 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:13.095732 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:13.095739 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:13.096428 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:13.096430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:13.097669 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:13.097673 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:13.098176 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:13.098177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:13.098314 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:13.098887 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:13.098897 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:13.100356 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:13.102419 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:13.102426 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:13.102428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:13.102700 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:13.102948 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:13.102949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:13.102949 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:13.103084 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:13.103377 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:13.103384 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:13.103537 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 23:02:13.103638 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 23:02:13.103802 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":79,"threads_started":1}
I20260808 23:02:13.104413 12197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:13.094022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:40143 (local address 127.23.194.62:36701)
0808 23:02:13.094104 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:13.094106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:13.094304 (+ 198us) server_negotiation.cc:407] Connection header received
0808 23:02:13.094980 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:13.094981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:13.095007 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:13.095037 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:13.095819 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:13.096334 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:13.097768 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:13.098053 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:13.098132 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:13.100463 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:13.100464 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:13.100465 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:13.100472 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:13.102189 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:13.102791 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:13.102793 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:13.102793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:13.102862 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:13.103158 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:13.103159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:13.103159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:13.103294 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:13.103352 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:13.104182 (+ 830us) server_negotiation.cc:299] Negotiation successful
0808 23:02:13.104320 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 23:02:13.104787 12622 heartbeater.cc:347] Connected to a master server at 127.23.194.62:36701
I20260808 23:02:13.104887 12622 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:13.105068 12622 heartbeater.cc:511] Master 127.23.194.62:36701 requested a full tablet report, sending...
W20260808 23:02:13.105718 12622 heartbeater.cc:644] Failed to heartbeat to 127.23.194.62:36701 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 23:02:13.105846 12622 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:13.105949 12622 heartbeater.cc:511] Master 127.23.194.62:36701 requested a full tablet report, sending...
I20260808 23:02:13.106701 12622 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:13.106832 12622 heartbeater.cc:511] Master 127.23.194.62:36701 requested a full tablet report, sending...
W20260808 23:02:13.107362 12622 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 23:02:13.110560 12626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:13.102325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40372 (local address 127.23.194.62:36701)
0808 23:02:13.102825 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:13.102827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:13.103040 (+ 213us) server_negotiation.cc:407] Connection header received
0808 23:02:13.103061 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:13.103062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:13.103074 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:13.103134 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:13.103891 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:13.105307 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:13.106246 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:13.106554 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:13.106631 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:13.107921 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:13.107923 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:13.107923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:13.107930 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:13.109483 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:13.109800 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:13.109801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:13.109802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:13.109857 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:13.110037 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:13.110038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:13.110038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:13.110169 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:13.110341 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:13.110384 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 23:02:13.110465 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":57,"threads_started":1}
Aug 08 23:02:13 dist-test-slave-f0tn krb5kdc[10278](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786230133, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:13.767549 12188 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:13.767715 12188 heartbeater.cc:511] Master 127.23.194.62:36701 requested a full tablet report, sending...
I20260808 23:02:13.768126 11852 ts_manager.cc:194] Registered new tserver with Master: 5dcdb9da52b1488c96297576ccab4cc3 (127.23.194.1:44637)
I20260808 23:02:13.769127 11852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:44987
I20260808 23:02:13.925276 12404 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:13.925459 12404 heartbeater.cc:511] Master 127.23.194.62:36701 requested a full tablet report, sending...
I20260808 23:02:13.925776 11852 ts_manager.cc:194] Registered new tserver with Master: c67d584ecea04738bef997b6b44c2a41 (127.23.194.2:36091)
I20260808 23:02:13.926314 11852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:34919
Aug 08 23:02:13 dist-test-slave-f0tn krb5kdc[10278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230133, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for HTTP/127.23.194.20@KRBTEST.COM
I20260808 23:02:14.107532 12622 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:14.107676 12622 heartbeater.cc:511] Master 127.23.194.62:36701 requested a full tablet report, sending...
I20260808 23:02:14.107980 11852 ts_manager.cc:194] Registered new tserver with Master: f35dcf39324944d495aed69610537c95 (127.23.194.3:33713)
I20260808 23:02:14.108472 11852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:40143
I20260808 23:02:14.112447 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:02:14.125144 12651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:14.113474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40378 (local address 127.23.194.62:36701)
0808 23:02:14.117518 (+ 4044us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:14.117520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:14.117532 (+ 12us) server_negotiation.cc:407] Connection header received
0808 23:02:14.117560 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:14.117560 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:14.117574 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:14.117668 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:14.118441 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.119218 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.120214 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.120550 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.120634 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:14.122090 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:14.122092 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:14.122093 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:14.122101 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:14.123880 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:14.124286 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:14.124287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:14.124287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:14.124364 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:14.124569 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:14.124570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:14.124571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:14.124720 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:14.124838 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:14.124935 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 23:02:14.125030 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4016,"thread_start_us":67,"threads_started":1}
I20260808 23:02:14.127763 11852 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:02:14.130578 11852 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:02:14.143748 12662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:14.138061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:36091 (local address 127.0.0.1:53342)
0808 23:02:14.138556 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:14.139034 (+ 478us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:14.139109 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:14.139708 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:14.139709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:14.139712 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:02:14.139749 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:14.139752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.141070 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.141071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:14.142227 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:14.142230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.143373 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.143375 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:14.143463 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:14.143479 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 23:02:14.143564 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 23:02:14.143633 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":97,"threads_started":1}
I20260808 23:02:14.144104 12666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:14.138360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40148 (local address 127.23.194.3:33713)
0808 23:02:14.138990 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:14.138996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:14.139203 (+ 207us) server_negotiation.cc:407] Connection header received
0808 23:02:14.139245 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:14.139247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:14.139260 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:14.139334 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:02:14.139553 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.140655 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.142682 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.143488 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.143562 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:14.143619 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:14.143975 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 23:02:14.144021 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":62,"threads_started":1}
I20260808 23:02:14.143875 12664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:14.138061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:33713 (local address 127.0.0.1:40148)
0808 23:02:14.138971 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:14.139033 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:14.139109 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:14.139355 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:14.139359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:14.139361 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:02:14.139450 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:14.139452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.140770 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.140772 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:14.142577 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:14.142579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.143657 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.143658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:14.143718 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:14.143721 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:02:14.143774 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 23:02:14.143812 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":5,"thread_start_us":99,"threads_started":1}
I20260808 23:02:14.145437 12511 tablet_service.cc:1467] Processing CreateTablet for tablet 54ee440dba9a43dc8aa48783ff03b5e7 (DEFAULT_TABLE table=test-table [id=167562e14fb94da1a99400ad81d7dae0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:02:14.145882 12511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54ee440dba9a43dc8aa48783ff03b5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:02:14.148623 12662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:14.138308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:44637 (local address 127.0.0.1:34486)
0808 23:02:14.144352 (+ 6044us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:14.144363 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:14.144419 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:14.144720 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:14.144721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:14.144723 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:02:14.144760 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:14.144762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.146331 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.146332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:14.147330 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:14.147333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.148368 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.148369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:14.148428 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:14.148432 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:02:14.148499 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 23:02:14.148546 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6029,"thread_start_us":72,"threads_started":1}
I20260808 23:02:14.148880 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:14.138808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34486 (local address 127.23.194.1:44637)
0808 23:02:14.139884 (+ 1076us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:14.139888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:14.144448 (+ 4560us) server_negotiation.cc:407] Connection header received
0808 23:02:14.144508 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:14.144511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:14.144532 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:14.144616 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:02:14.144846 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.146222 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.147442 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.148192 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.148270 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:14.148329 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:14.148749 (+ 420us) server_negotiation.cc:299] Negotiation successful
0808 23:02:14.148793 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1043,"thread_start_us":68,"threads_started":1}
I20260808 23:02:14.149667 12071 tablet_service.cc:1467] Processing CreateTablet for tablet 54ee440dba9a43dc8aa48783ff03b5e7 (DEFAULT_TABLE table=test-table [id=167562e14fb94da1a99400ad81d7dae0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:02:14.150017 12071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54ee440dba9a43dc8aa48783ff03b5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:02:14.155130 12667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:14.138609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53342 (local address 127.23.194.2:36091)
0808 23:02:14.139479 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:14.139485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:14.139495 (+ 10us) server_negotiation.cc:407] Connection header received
0808 23:02:14.139528 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:14.139530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:14.139546 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:14.139623 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:02:14.139921 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.140968 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.142335 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.143181 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.143265 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:14.143331 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:14.154947 (+ 11616us) server_negotiation.cc:299] Negotiation successful
0808 23:02:14.155038 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":834,"thread_start_us":92,"threads_started":1}
I20260808 23:02:14.156036 12299 tablet_service.cc:1467] Processing CreateTablet for tablet 54ee440dba9a43dc8aa48783ff03b5e7 (DEFAULT_TABLE table=test-table [id=167562e14fb94da1a99400ad81d7dae0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:02:14.156381 12299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54ee440dba9a43dc8aa48783ff03b5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:02:14.155596 12672 tablet_bootstrap.cc:492] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95: Bootstrap starting.
I20260808 23:02:14.158998 12671 tablet_bootstrap.cc:492] T 54ee440dba9a43dc8aa48783ff03b5e7 P 5dcdb9da52b1488c96297576ccab4cc3: Bootstrap starting.
I20260808 23:02:14.159776 12671 tablet_bootstrap.cc:654] T 54ee440dba9a43dc8aa48783ff03b5e7 P 5dcdb9da52b1488c96297576ccab4cc3: Neither blocks nor log segments found. Creating new log.
I20260808 23:02:14.158691 12672 tablet_bootstrap.cc:654] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95: Neither blocks nor log segments found. Creating new log.
I20260808 23:02:14.160288 12672 log.cc:824] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:14.160676 12671 log.cc:824] T 54ee440dba9a43dc8aa48783ff03b5e7 P 5dcdb9da52b1488c96297576ccab4cc3: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:14.161952 12672 tablet_bootstrap.cc:492] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95: No bootstrap required, opened a new log
I20260808 23:02:14.162039 12672 ts_tablet_manager.cc:1397] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260808 23:02:14.164645 12672 raft_consensus.cc:348] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35dcf39324944d495aed69610537c95" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33713 } } peers { permanent_uuid: "c67d584ecea04738bef997b6b44c2a41" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36091 } } peers { permanent_uuid: "5dcdb9da52b1488c96297576ccab4cc3" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 44637 } }
I20260808 23:02:14.164817 12672 raft_consensus.cc:374] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:02:14.164848 12672 raft_consensus.cc:729] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35dcf39324944d495aed69610537c95, State: Initialized, Role: FOLLOWER
I20260808 23:02:14.164980 12672 consensus_queue.cc:260] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35dcf39324944d495aed69610537c95" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33713 } } peers { permanent_uuid: "c67d584ecea04738bef997b6b44c2a41" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36091 } } peers { permanent_uuid: "5dcdb9da52b1488c96297576ccab4cc3" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 44637 } }
I20260808 23:02:14.165525 12674 tablet_bootstrap.cc:492] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41: Bootstrap starting.
I20260808 23:02:14.165750 12671 tablet_bootstrap.cc:492] T 54ee440dba9a43dc8aa48783ff03b5e7 P 5dcdb9da52b1488c96297576ccab4cc3: No bootstrap required, opened a new log
I20260808 23:02:14.165864 12671 ts_tablet_manager.cc:1397] T 54ee440dba9a43dc8aa48783ff03b5e7 P 5dcdb9da52b1488c96297576ccab4cc3: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260808 23:02:14.166491 12674 tablet_bootstrap.cc:654] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41: Neither blocks nor log segments found. Creating new log.
I20260808 23:02:14.167075 12674 log.cc:824] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:14.169193 12671 raft_consensus.cc:348] T 54ee440dba9a43dc8aa48783ff03b5e7 P 5dcdb9da52b1488c96297576ccab4cc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35dcf39324944d495aed69610537c95" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33713 } } peers { permanent_uuid: "c67d584ecea04738bef997b6b44c2a41" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36091 } } peers { permanent_uuid: "5dcdb9da52b1488c96297576ccab4cc3" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 44637 } }
I20260808 23:02:14.169481 12672 ts_tablet_manager.cc:1428] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260808 23:02:14.169629 12622 heartbeater.cc:503] Master 127.23.194.62:36701 was elected leader, sending a full tablet report...
I20260808 23:02:14.170244 12674 tablet_bootstrap.cc:492] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41: No bootstrap required, opened a new log
I20260808 23:02:14.170323 12674 ts_tablet_manager.cc:1397] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 23:02:14.170620 12671 raft_consensus.cc:374] T 54ee440dba9a43dc8aa48783ff03b5e7 P 5dcdb9da52b1488c96297576ccab4cc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:02:14.170717 12671 raft_consensus.cc:729] T 54ee440dba9a43dc8aa48783ff03b5e7 P 5dcdb9da52b1488c96297576ccab4cc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dcdb9da52b1488c96297576ccab4cc3, State: Initialized, Role: FOLLOWER
I20260808 23:02:14.170887 12671 consensus_queue.cc:260] T 54ee440dba9a43dc8aa48783ff03b5e7 P 5dcdb9da52b1488c96297576ccab4cc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35dcf39324944d495aed69610537c95" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33713 } } peers { permanent_uuid: "c67d584ecea04738bef997b6b44c2a41" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36091 } } peers { permanent_uuid: "5dcdb9da52b1488c96297576ccab4cc3" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 44637 } }
I20260808 23:02:14.171196 12671 ts_tablet_manager.cc:1428] T 54ee440dba9a43dc8aa48783ff03b5e7 P 5dcdb9da52b1488c96297576ccab4cc3: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 23:02:14.171345 12188 heartbeater.cc:503] Master 127.23.194.62:36701 was elected leader, sending a full tablet report...
I20260808 23:02:14.172600 12674 raft_consensus.cc:348] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35dcf39324944d495aed69610537c95" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33713 } } peers { permanent_uuid: "c67d584ecea04738bef997b6b44c2a41" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36091 } } peers { permanent_uuid: "5dcdb9da52b1488c96297576ccab4cc3" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 44637 } }
I20260808 23:02:14.172835 12674 raft_consensus.cc:374] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:02:14.173026 12674 raft_consensus.cc:729] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c67d584ecea04738bef997b6b44c2a41, State: Initialized, Role: FOLLOWER
I20260808 23:02:14.173233 12674 consensus_queue.cc:260] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35dcf39324944d495aed69610537c95" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33713 } } peers { permanent_uuid: "c67d584ecea04738bef997b6b44c2a41" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36091 } } peers { permanent_uuid: "5dcdb9da52b1488c96297576ccab4cc3" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 44637 } }
I20260808 23:02:14.173604 12674 ts_tablet_manager.cc:1428] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 23:02:14.173766 12404 heartbeater.cc:503] Master 127.23.194.62:36701 was elected leader, sending a full tablet report...
W20260808 23:02:14.235595 12189 tablet.cc:2369] T 54ee440dba9a43dc8aa48783ff03b5e7 P 5dcdb9da52b1488c96297576ccab4cc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 23:02:14 dist-test-slave-f0tn krb5kdc[10278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230133, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for HTTP/127.23.194.20@KRBTEST.COM
I20260808 23:02:14.239321 12678 raft_consensus.cc:482] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:02:14.239482 12678 raft_consensus.cc:504] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35dcf39324944d495aed69610537c95" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33713 } } peers { permanent_uuid: "c67d584ecea04738bef997b6b44c2a41" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36091 } } peers { permanent_uuid: "5dcdb9da52b1488c96297576ccab4cc3" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 44637 } }
I20260808 23:02:14.239880 12678 leader_election.cc:288] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c67d584ecea04738bef997b6b44c2a41 (127.23.194.2:36091), 5dcdb9da52b1488c96297576ccab4cc3 (127.23.194.1:44637)
I20260808 23:02:14.244256 12667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:14.240428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:48741 (local address 127.23.194.2:36091)
0808 23:02:14.240535 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:14.240538 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:14.240696 (+ 158us) server_negotiation.cc:407] Connection header received
0808 23:02:14.240802 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:14.240803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:14.240813 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:14.240842 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:02:14.241133 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.241834 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.243357 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.243847 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.243894 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:14.243904 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:14.244123 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 23:02:14.244180 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 23:02:14.244525 12687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:14.240019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:36091 (local address 127.23.194.3:48741)
0808 23:02:14.240632 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:14.240642 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:14.240726 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:14.240913 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:14.240914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:14.240916 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:02:14.241054 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:14.241057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.241937 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.241938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:14.243257 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:14.243261 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.243980 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.243981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:14.244042 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:14.244046 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:02:14.244382 (+ 336us) client_negotiation.cc:240] Negotiation successful
0808 23:02:14.244429 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":96,"threads_started":1}
I20260808 23:02:14.244869 12328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54ee440dba9a43dc8aa48783ff03b5e7" candidate_uuid: "f35dcf39324944d495aed69610537c95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c67d584ecea04738bef997b6b44c2a41" is_pre_election: true
I20260808 23:02:14.245042 12328 raft_consensus.cc:2454] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f35dcf39324944d495aed69610537c95 in term 0.
I20260808 23:02:14.245299 12439 leader_election.cc:302] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c67d584ecea04738bef997b6b44c2a41, f35dcf39324944d495aed69610537c95; no voters:
I20260808 23:02:14.245464 12678 raft_consensus.cc:2781] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:02:14.245520 12678 raft_consensus.cc:482] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:02:14.245543 12678 raft_consensus.cc:3037] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:02:14.246465 12678 raft_consensus.cc:504] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35dcf39324944d495aed69610537c95" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33713 } } peers { permanent_uuid: "c67d584ecea04738bef997b6b44c2a41" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36091 } } peers { permanent_uuid: "5dcdb9da52b1488c96297576ccab4cc3" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 44637 } }
I20260808 23:02:14.246595 12678 leader_election.cc:288] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [CANDIDATE]: Term 1 election: Requested vote from peers c67d584ecea04738bef997b6b44c2a41 (127.23.194.2:36091), 5dcdb9da52b1488c96297576ccab4cc3 (127.23.194.1:44637)
I20260808 23:02:14.246778 12328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54ee440dba9a43dc8aa48783ff03b5e7" candidate_uuid: "f35dcf39324944d495aed69610537c95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c67d584ecea04738bef997b6b44c2a41"
I20260808 23:02:14.246860 12328 raft_consensus.cc:3037] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:02:14.247637 12328 raft_consensus.cc:2454] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f35dcf39324944d495aed69610537c95 in term 1.
I20260808 23:02:14.247805 12439 leader_election.cc:302] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c67d584ecea04738bef997b6b44c2a41, f35dcf39324944d495aed69610537c95; no voters:
I20260808 23:02:14.247880 12678 raft_consensus.cc:2781] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:02:14.248027 12678 raft_consensus.cc:686] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [term 1 LEADER]: Becoming Leader. State: Replica: f35dcf39324944d495aed69610537c95, State: Running, Role: LEADER
I20260808 23:02:14.248126 12678 consensus_queue.cc:237] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35dcf39324944d495aed69610537c95" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33713 } } peers { permanent_uuid: "c67d584ecea04738bef997b6b44c2a41" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36091 } } peers { permanent_uuid: "5dcdb9da52b1488c96297576ccab4cc3" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 44637 } }
I20260808 23:02:14.248900 11852 catalog_manager.cc:5495] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 reported cstate change: term changed from 0 to 1, leader changed from <none> to f35dcf39324944d495aed69610537c95 (127.23.194.3). New cstate: current_term: 1 leader_uuid: "f35dcf39324944d495aed69610537c95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35dcf39324944d495aed69610537c95" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c67d584ecea04738bef997b6b44c2a41" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dcdb9da52b1488c96297576ccab4cc3" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 44637 } health_report { overall_health: UNKNOWN } } }
I20260808 23:02:14.252677 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:14.240491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:54509 (local address 127.23.194.1:44637)
0808 23:02:14.240567 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:14.240569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:14.249462 (+ 8893us) server_negotiation.cc:407] Connection header received
0808 23:02:14.249554 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:14.249555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:14.249566 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:14.249595 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:02:14.249771 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.250428 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.251753 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.252333 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.252382 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:14.252390 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:14.252574 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 23:02:14.252612 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 23:02:14.252897 12687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:14.240218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:44637 (local address 127.23.194.3:54509)
0808 23:02:14.249400 (+ 9182us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:14.249413 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:14.249487 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:14.249658 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:14.249659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:14.249660 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:02:14.249691 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:14.249692 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.250587 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.250588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:14.251648 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:14.251651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.252445 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.252446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:14.252504 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:14.252507 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:02:14.252798 (+ 291us) client_negotiation.cc:240] Negotiation successful
0808 23:02:14.252837 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9151,"thread_start_us":65,"threads_started":1}
I20260808 23:02:14.253197 12102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54ee440dba9a43dc8aa48783ff03b5e7" candidate_uuid: "f35dcf39324944d495aed69610537c95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dcdb9da52b1488c96297576ccab4cc3" is_pre_election: true
I20260808 23:02:14.253216 12101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54ee440dba9a43dc8aa48783ff03b5e7" candidate_uuid: "f35dcf39324944d495aed69610537c95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dcdb9da52b1488c96297576ccab4cc3"
I20260808 23:02:14.253351 12102 raft_consensus.cc:2454] T 54ee440dba9a43dc8aa48783ff03b5e7 P 5dcdb9da52b1488c96297576ccab4cc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f35dcf39324944d495aed69610537c95 in term 0.
I20260808 23:02:14.283145 12666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:14.279946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40164 (local address 127.23.194.3:33713)
0808 23:02:14.279995 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:14.279997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:14.280006 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:02:14.280045 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:14.280046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:14.280062 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:14.280125 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:02:14.280307 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.280966 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.282018 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:14.282283 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:14.282352 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:14.282581 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:02:14.282729 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:02:14.282832 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:14.283041 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 23:02:14.283066 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 23:02:14.286345 12101 raft_consensus.cc:3037] T 54ee440dba9a43dc8aa48783ff03b5e7 P 5dcdb9da52b1488c96297576ccab4cc3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:02:14.287284 12101 raft_consensus.cc:1261] T 54ee440dba9a43dc8aa48783ff03b5e7 P 5dcdb9da52b1488c96297576ccab4cc3 [term 1 FOLLOWER]: Refusing update from remote peer f35dcf39324944d495aed69610537c95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:02:14.287616 12678 consensus_queue.cc:1035] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dcdb9da52b1488c96297576ccab4cc3" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 44637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:02:14.288132 12328 raft_consensus.cc:1261] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41 [term 1 FOLLOWER]: Refusing update from remote peer f35dcf39324944d495aed69610537c95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:02:14.288532 12688 consensus_queue.cc:1035] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c67d584ecea04738bef997b6b44c2a41" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 36091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:02:14.295454 11852 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:02:14.295558 11852 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:02:14.296411 11852 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d71fdef57f394e6c850581f17e5ef83a: Sending DeleteTablet for 3 replicas of tablet 54ee440dba9a43dc8aa48783ff03b5e7
I20260808 23:02:14.296932 12511 tablet_service.cc:1514] Processing DeleteTablet for tablet 54ee440dba9a43dc8aa48783ff03b5e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:02:14 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:40148
I20260808 23:02:14.297170 12299 tablet_service.cc:1514] Processing DeleteTablet for tablet 54ee440dba9a43dc8aa48783ff03b5e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:02:14 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:53342
I20260808 23:02:14.297425 12071 tablet_service.cc:1514] Processing DeleteTablet for tablet 54ee440dba9a43dc8aa48783ff03b5e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:02:14 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:34486
I20260808 23:02:14.298156 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 11923
I20260808 23:02:14.315953 12696 mvcc.cc:205] Tried to move back new op lower bound from 7316398630034432000 to 7316398629880954880. Current Snapshot: MvccSnapshot[applied={T|T < 7316398630034432000}]
I20260808 23:02:14.319249 12711 tablet_replica.cc:330] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95: stopping tablet replica
I20260808 23:02:14.319466 12714 tablet_replica.cc:330] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41: stopping tablet replica
I20260808 23:02:14.324090 12711 raft_consensus.cc:2229] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:02:14.324275 12714 raft_consensus.cc:2229] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:02:14.324410 12714 raft_consensus.cc:2258] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:02:14.324848 12711 raft_consensus.cc:2258] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:02:14.326046 12711 ts_tablet_manager.cc:1905] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:02:14.326469 12714 ts_tablet_manager.cc:1905] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:02:14.329761 12714 ts_tablet_manager.cc:1918] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:02:14.332392 12714 log.cc:1196] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/54ee440dba9a43dc8aa48783ff03b5e7
I20260808 23:02:14.332599 12711 ts_tablet_manager.cc:1918] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:02:14.333001 12714 ts_tablet_manager.cc:1939] T 54ee440dba9a43dc8aa48783ff03b5e7 P c67d584ecea04738bef997b6b44c2a41: Deleting consensus metadata
I20260808 23:02:14.333127 12711 log.cc:1196] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/54ee440dba9a43dc8aa48783ff03b5e7
I20260808 23:02:14.333479 12711 ts_tablet_manager.cc:1939] T 54ee440dba9a43dc8aa48783ff03b5e7 P f35dcf39324944d495aed69610537c95: Deleting consensus metadata
I20260808 23:02:14.334007 11841 catalog_manager.cc:4841] TS c67d584ecea04738bef997b6b44c2a41 (127.23.194.2:36091): tablet 54ee440dba9a43dc8aa48783ff03b5e7 (table test-table [id=167562e14fb94da1a99400ad81d7dae0]) successfully deleted
I20260808 23:02:14.334532 11843 catalog_manager.cc:4841] TS f35dcf39324944d495aed69610537c95 (127.23.194.3:33713): tablet 54ee440dba9a43dc8aa48783ff03b5e7 (table test-table [id=167562e14fb94da1a99400ad81d7dae0]) successfully deleted
W20260808 23:02:14.334930 11841 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.194.1:44637 (error 108)
I20260808 23:02:14.335839 12661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:14.335698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:44637 (local address 127.0.0.1:34500)
0808 23:02:14.335747 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:14.335794 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.1:44637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 23:02:14.335852 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 12199
W20260808 23:02:14.387318 11841 catalog_manager.cc:4584] TS 5dcdb9da52b1488c96297576ccab4cc3 (127.23.194.1:44637): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 54ee440dba9a43dc8aa48783ff03b5e7: Network error: Client connection negotiation failed: client connection to 127.23.194.1:44637: connect: Connection refused (error 111)
I20260808 23:02:14.393646 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 12410
I20260808 23:02:14.407688 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 11811
I20260808 23:02:14.419343 24328 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 23:02:14.766618 24328 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 23:02:14.766714 24328 mini_ranger.cc:66] Stopping Ranger...
I20260808 23:02:15.110471 24328 mini_ranger.cc:68] Stopped Ranger
2026-08-08 23:02:15.110 UTC [10326] LOG: received smart shutdown request
2026-08-08 23:02:15.114 UTC [10326] LOG: background worker "logical replication launcher" (PID 10340) exited with exit code 1
2026-08-08 23:02:15.115 UTC [10332] LOG: shutting down
2026-08-08 23:02:15.248 UTC [10326] LOG: database system is shut down
2026-08-08T23:02:15Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47273 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:02:15 dist-test-slave-f0tn krb5kdc[12740](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:02:15 dist-test-slave-f0tn krb5kdc[12740](info): set up 2 sockets
Aug 08 23:02:15 dist-test-slave-f0tn krb5kdc[12740](info): commencing operation
krb5kdc: starting...
W20260808 23:02:16.481920 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:02:16 dist-test-slave-f0tn krb5kdc[12740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:02:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:02:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:02:16.605966 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:46129
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:39381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:46129,127.23.194.61:38563,127.23.194.60:38857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:02:16.685714 12756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:16.685909 12756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:16.685930 12756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:16.687026 12756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:02:16.687059 12756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:16.687072 12756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:16.687083 12756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:02:16.687098 12756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:02:16.688532 12756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:39381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:46129,127.23.194.61:38563,127.23.194.60:38857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:46129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:16.688727 12756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:16.688894 12756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:16.693673 12763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:16.693686 12767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:16.693729 12756 server_base.cc:1034] running on GCE node
W20260808 23:02:16.693684 12765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:16.694068 12756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:16.694357 12756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:16.695499 12756 hybrid_clock.cc:648] HybridClock initialized: now 1786230136695485 us; error 26 us; skew 500 ppm
Aug 08 23:02:16 dist-test-slave-f0tn krb5kdc[12740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230136, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:16.700901 12756 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:02:16.701327 12756 webserver.cc:460] Webserver started at http://127.23.194.62:40655/ using document root <none> and password file <none>
I20260808 23:02:16.701537 12756 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:16.701581 12756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:16.701675 12756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:02:16.702663 12756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "14401145e7c44092b8581caf9a00d5c7"
format_stamp: "Formatted at 2026-08-08 23:02:16 on dist-test-slave-f0tn"
server_key: "0eb2dfdf8dabe66f3542bab0d95d46cf"
server_key_iv: "cc67c3cf8201df9daa914df2ca5af819"
server_key_version: "encryptionkey@0"
I20260808 23:02:16.702946 12756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "14401145e7c44092b8581caf9a00d5c7"
format_stamp: "Formatted at 2026-08-08 23:02:16 on dist-test-slave-f0tn"
server_key: "0eb2dfdf8dabe66f3542bab0d95d46cf"
server_key_iv: "cc67c3cf8201df9daa914df2ca5af819"
server_key_version: "encryptionkey@0"
I20260808 23:02:16.704106 12756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:02:16.704743 12777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:16.704923 12756 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:02:16.705006 12756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "14401145e7c44092b8581caf9a00d5c7"
format_stamp: "Formatted at 2026-08-08 23:02:16 on dist-test-slave-f0tn"
server_key: "0eb2dfdf8dabe66f3542bab0d95d46cf"
server_key_iv: "cc67c3cf8201df9daa914df2ca5af819"
server_key_version: "encryptionkey@0"
I20260808 23:02:16.705060 12756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:16.719063 12756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:16.721278 12756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:16.721418 12756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:16.725353 12756 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:46129
I20260808 23:02:16.725411 12859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:46129 every 8 connection(s)
I20260808 23:02:16.725816 12756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:02:16.726811 12861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:02:16.728271 12861 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:02:16.730132 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 12756
I20260808 23:02:16.730244 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:02:16.730423 24328 external_mini_cluster.cc:1428] Setting key 2498f5f5a781cc451f68909af3776ce5
Aug 08 23:02:16 dist-test-slave-f0tn krb5kdc[12740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230136, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:16.737540 12864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.728999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58176 (local address 127.23.194.62:46129)
0808 23:02:16.729150 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:16.729160 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:16.729261 (+ 101us) server_negotiation.cc:407] Connection header received
0808 23:02:16.729815 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:16.729817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:16.729834 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:16.729908 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:16.730467 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:16.731297 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:16.732290 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:16.732608 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:16.732685 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:16.734710 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:16.734717 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:16.734717 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:16.734723 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:16.736082 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:16.736497 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:16.736499 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:16.736499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:16.736567 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:16.736794 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:16.736795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:16.736795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:16.736973 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:16.737029 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:16.737237 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 23:02:16.737357 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
I20260808 23:02:16.737610 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.728951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:46129 (local address 127.0.0.1:58176)
0808 23:02:16.729116 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:16.729171 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:16.729671 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:16.729926 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:16.729928 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:16.730185 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:16.730379 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:16.730384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:16.731392 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:16.731394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:16.732191 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:16.732194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:16.732689 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:16.732690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:16.732765 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:16.733195 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:16.733203 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:16.734628 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:16.736168 (+ 1540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:16.736175 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:16.736177 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:16.736426 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:16.736625 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:16.736626 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:16.736626 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:16.736736 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:16.737038 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:16.737045 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:16.737118 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 23:02:16.737219 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 23:02:16.737465 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":94,"thread_start_us":73,"threads_started":1}
I20260808 23:02:16.738297 12861 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.194.61@KRBTEST.COM; defaulting to no policy
I20260808 23:02:16.738588 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.738477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:38563 (local address 127.0.0.1:38374)
0808 23:02:16.738511 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:16.738562 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.61:38563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260808 23:02:16.738720 12779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.194.61:38563: connect: Connection refused (error 111)
I20260808 23:02:16.739135 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.739036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:38563 (local address 127.0.0.1:38380)
0808 23:02:16.739088 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:16.739114 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.61:38563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 23:02:16.739241 12861 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.61:38563: Network error: Client connection negotiation failed: client connection to 127.23.194.61:38563: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.61.keytab.
Entry for principal kudu/127.23.194.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.61.keytab.
I20260808 23:02:16.753695 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.61:38563
--webserver_interface=127.23.194.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.61.keytab
--principal=kudu/127.23.194.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:39381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:46129,127.23.194.61:38563,127.23.194.60:38857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:02:16.788494 12861 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } attempt: 1
I20260808 23:02:16.789055 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.788922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:38563 (local address 127.0.0.1:38384)
0808 23:02:16.788987 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:16.789021 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.61:38563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 23:02:16.789428 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.789318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:38563 (local address 127.0.0.1:38394)
0808 23:02:16.789352 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:16.789399 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.61:38563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260808 23:02:16.789525 12861 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.61:38563: Network error: Client connection negotiation failed: client connection to 127.23.194.61:38563: connect: Connection refused (error 111)
W20260808 23:02:16.835747 12868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:16.835956 12868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:16.835986 12868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:16.837108 12868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:02:16.837148 12868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:16.837167 12868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:16.837183 12868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:02:16.837198 12868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:02:16.838712 12868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:39381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:46129,127.23.194.61:38563,127.23.194.60:38857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.61.keytab
--principal=kudu/127.23.194.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.61:38563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.194.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:16.838958 12868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:16.839167 12868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:16.844197 12875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:16.844213 12876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:16.844218 12881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:16.844292 12868 server_base.cc:1034] running on GCE node
I20260808 23:02:16.844633 12868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:16.844918 12868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:16.846076 12868 hybrid_clock.cc:648] HybridClock initialized: now 1786230136846058 us; error 36 us; skew 500 ppm
Aug 08 23:02:16 dist-test-slave-f0tn krb5kdc[12740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230136, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:16.848438 12868 init.cc:377] Logged in from keytab as kudu/127.23.194.61@KRBTEST.COM (short username kudu)
I20260808 23:02:16.848860 12868 webserver.cc:460] Webserver started at http://127.23.194.61:36429/ using document root <none> and password file <none>
I20260808 23:02:16.849035 12868 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:16.849074 12868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:16.849169 12868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:02:16.850188 12868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-1/data/instance:
uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c"
format_stamp: "Formatted at 2026-08-08 23:02:16 on dist-test-slave-f0tn"
server_key: "b4fbbd60e7388fe8bb90af1694e5cb4b"
server_key_iv: "bcd29109a7e76ef3ed281b2631b74667"
server_key_version: "encryptionkey@0"
I20260808 23:02:16.850473 12868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-1/wal/instance:
uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c"
format_stamp: "Formatted at 2026-08-08 23:02:16 on dist-test-slave-f0tn"
server_key: "b4fbbd60e7388fe8bb90af1694e5cb4b"
server_key_iv: "bcd29109a7e76ef3ed281b2631b74667"
server_key_version: "encryptionkey@0"
I20260808 23:02:16.851683 12868 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 23:02:16.852401 12893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:16.852559 12868 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:02:16.852645 12868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-1/wal
uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c"
format_stamp: "Formatted at 2026-08-08 23:02:16 on dist-test-slave-f0tn"
server_key: "b4fbbd60e7388fe8bb90af1694e5cb4b"
server_key_iv: "bcd29109a7e76ef3ed281b2631b74667"
server_key_version: "encryptionkey@0"
I20260808 23:02:16.852700 12868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:16.857659 12861 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } attempt: 2
I20260808 23:02:16.858057 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.857924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:38563 (local address 127.0.0.1:38404)
0808 23:02:16.857983 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:16.858026 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.61:38563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 23:02:16.858351 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.858284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:38563 (local address 127.0.0.1:38412)
0808 23:02:16.858314 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:16.858336 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.61:38563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260808 23:02:16.858431 12861 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.61:38563: Network error: Client connection negotiation failed: client connection to 127.23.194.61:38563: connect: Connection refused (error 111)
I20260808 23:02:16.866918 12868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:16.869119 12868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:16.869238 12868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:16.873238 12868 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.61:38563
I20260808 23:02:16.873279 12967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.61:38563 every 8 connection(s)
I20260808 23:02:16.873643 12868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-1/data/info.pb
I20260808 23:02:16.874552 12968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:02:16.876088 12968 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:02:16.878216 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 12868
I20260808 23:02:16.878333 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-1/wal/instance
I20260808 23:02:16.878511 24328 external_mini_cluster.cc:1428] Setting key 9ed1974acd12a5c291ba853cbecfe161
Aug 08 23:02:16 dist-test-slave-f0tn krb5kdc[12740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230136, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.61@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:16.885063 12864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.876762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58192 (local address 127.23.194.62:46129)
0808 23:02:16.876838 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:16.876840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:16.876993 (+ 153us) server_negotiation.cc:407] Connection header received
0808 23:02:16.877446 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:16.877447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:16.877463 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:16.877520 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:16.878058 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:16.878774 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:16.879771 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:16.880050 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:16.880125 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:16.882536 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:16.882537 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:16.882538 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:16.882545 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:16.883824 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:16.884187 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:16.884188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:16.884189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:16.884243 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:16.884455 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:16.884455 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:16.884456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:16.884566 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:16.884620 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:16.884886 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 23:02:16.884973 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 23:02:16.885217 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.876715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:46129 (local address 127.0.0.1:58192)
0808 23:02:16.876865 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:16.876905 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:16.877347 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:16.877569 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:16.877572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:16.877790 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:16.877967 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:16.877971 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:16.878884 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:16.878885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:16.879682 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:16.879685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:16.880160 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:16.880162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:16.880323 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:16.880826 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:16.880836 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:16.882447 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:16.883901 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:16.883908 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:16.883911 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:16.884117 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:16.884297 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:16.884297 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:16.884298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:16.884399 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:16.884619 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:16.884626 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:16.884771 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 23:02:16.884857 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 23:02:16.885005 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260808 23:02:16.885747 12968 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.194.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.60@KRBTEST.COM" created.
Aug 08 23:02:16 dist-test-slave-f0tn krb5kdc[12740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230136, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.61@KRBTEST.COM for kudu/127.23.194.61@KRBTEST.COM
I20260808 23:02:16.892908 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.886006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:38563 (local address 127.0.0.1:38426)
0808 23:02:16.886059 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:16.886072 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:16.886122 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:16.886356 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:16.886357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:16.886498 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:16.886559 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:16.886561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:16.887414 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:16.887415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:16.887927 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:16.887929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:16.888405 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:16.888406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:16.888492 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:16.888837 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:16.888842 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:16.890202 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:16.891909 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:16.891910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:16.891910 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:16.892076 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:16.892275 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:16.892275 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:16.892275 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:16.892322 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:16.892610 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:16.892611 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:16.892649 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 23:02:16.892712 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 23:02:16.892810 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 23:02:16.893069 12973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.886068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38426 (local address 127.23.194.61:38563)
0808 23:02:16.886224 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:16.886228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:16.886246 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:02:16.886265 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:16.886267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:16.886277 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:16.886334 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:16.886626 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:16.887338 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:16.887997 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:16.888312 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:16.888368 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:16.890272 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:16.890280 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:16.890281 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:16.890288 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:16.891826 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:16.892152 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:16.892155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:16.892155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:16.892219 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:16.892374 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:16.892374 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:16.892375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:16.892545 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:16.892621 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:16.892740 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 23:02:16.892881 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260808 23:02:16.893637 12968 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:02:16.893966 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.893861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:38857 (local address 127.0.0.1:47240)
0808 23:02:16.893899 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:16.893944 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:38857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 23:02:16.894090 12897 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.194.60:38857: connect: Connection refused (error 111)
I20260808 23:02:16.894495 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.894433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:38857 (local address 127.0.0.1:47242)
0808 23:02:16.894457 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:16.894480 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:38857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260808 23:02:16.894593 12968 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.60:38857: Network error: Client connection negotiation failed: client connection to 127.23.194.60:38857: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.60.keytab.
Entry for principal kudu/127.23.194.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.60.keytab.
I20260808 23:02:16.904438 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.60:38857
--webserver_interface=127.23.194.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.60.keytab
--principal=kudu/127.23.194.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:39381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:46129,127.23.194.61:38563,127.23.194.60:38857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:02:16.943842 12968 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } attempt: 1
I20260808 23:02:16.944397 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.944275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:38857 (local address 127.0.0.1:47248)
0808 23:02:16.944332 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:16.944364 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:38857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 23:02:16.944835 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.944725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:38857 (local address 127.0.0.1:47264)
0808 23:02:16.944761 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:16.944793 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:38857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260808 23:02:16.944937 12968 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.60:38857: Network error: Client connection negotiation failed: client connection to 127.23.194.60:38857: connect: Connection refused (error 111)
I20260808 23:02:16.949584 12861 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } attempt: 3
Aug 08 23:02:16 dist-test-slave-f0tn krb5kdc[12740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230136, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for kudu/127.23.194.61@KRBTEST.COM
I20260808 23:02:16.956525 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.949862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:38563 (local address 127.0.0.1:38436)
0808 23:02:16.949923 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:16.949936 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:16.949996 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:16.950144 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:16.950145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:16.950341 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:16.950427 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:16.950430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:16.951088 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:16.951089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:16.951875 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:16.951877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:16.952217 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:16.952218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:16.952296 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:16.952710 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:16.952715 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:16.954058 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:16.955578 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:16.955578 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:16.955579 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:16.955770 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:16.955990 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:16.955991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:16.955991 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:16.956055 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:16.956299 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:16.956300 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:16.956336 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 23:02:16.956385 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 23:02:16.956461 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 23:02:16.956600 12973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.949934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38436 (local address 127.23.194.61:38563)
0808 23:02:16.949981 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:16.949983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:16.949994 (+ 11us) server_negotiation.cc:407] Connection header received
0808 23:02:16.950066 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:16.950067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:16.950076 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:16.950130 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:16.950499 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:16.951012 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:16.951957 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:16.952161 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:16.952205 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:16.954140 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:16.954142 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:16.954142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:16.954150 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:16.955499 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:16.955859 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:16.955860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:16.955861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:16.955933 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:16.956116 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:16.956117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:16.956117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:16.956247 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:16.956309 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:16.956419 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 23:02:16.956510 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 23:02:16.956923 12861 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:02:16.957237 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.957145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:38857 (local address 127.0.0.1:47270)
0808 23:02:16.957185 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:16.957211 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:38857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 23:02:16.957537 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.957471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:38857 (local address 127.0.0.1:47282)
0808 23:02:16.957500 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:16.957522 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:38857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260808 23:02:16.957610 12861 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.60:38857: Network error: Client connection negotiation failed: client connection to 127.23.194.60:38857: connect: Connection refused (error 111)
I20260808 23:02:16.988734 12861 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } attempt: 1
I20260808 23:02:16.989231 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.989107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:38857 (local address 127.0.0.1:47288)
0808 23:02:16.989169 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:16.989206 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:38857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 23:02:16.989566 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:16.989494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:38857 (local address 127.0.0.1:47296)
0808 23:02:16.989529 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:16.989551 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:38857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 23:02:16.989656 12861 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.60:38857: Network error: Client connection negotiation failed: client connection to 127.23.194.60:38857: connect: Connection refused (error 111)
W20260808 23:02:16.990398 12985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:16.990571 12985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:16.990594 12985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:16.991662 12985 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:02:16.991694 12985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:16.991707 12985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:16.991719 12985 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:02:16.991729 12985 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:02:16.993175 12985 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:39381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:46129,127.23.194.61:38563,127.23.194.60:38857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.60.keytab
--principal=kudu/127.23.194.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.60:38857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.194.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:16.993358 12985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:16.993573 12985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:16.998278 12994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:16.998365 12995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:16.998291 12997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:16.998572 12985 server_base.cc:1034] running on GCE node
I20260808 23:02:16.998765 12985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:16.998991 12985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:17.000168 12985 hybrid_clock.cc:648] HybridClock initialized: now 1786230137000152 us; error 29 us; skew 500 ppm
Aug 08 23:02:16 dist-test-slave-f0tn krb5kdc[12740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230136, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:17.002645 12985 init.cc:377] Logged in from keytab as kudu/127.23.194.60@KRBTEST.COM (short username kudu)
I20260808 23:02:17.003023 12985 webserver.cc:460] Webserver started at http://127.23.194.60:35337/ using document root <none> and password file <none>
I20260808 23:02:17.003226 12985 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:17.003268 12985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:17.003357 12985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:02:17.004317 12985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-2/data/instance:
uuid: "46306eb2525c4adca1ff51587ee177c9"
format_stamp: "Formatted at 2026-08-08 23:02:16 on dist-test-slave-f0tn"
server_key: "6cf91eacfdb83db43b5da52c667b0555"
server_key_iv: "8d75baef9c1c5f8a8c20fc5afb37c83a"
server_key_version: "encryptionkey@0"
I20260808 23:02:17.004611 12985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-2/wal/instance:
uuid: "46306eb2525c4adca1ff51587ee177c9"
format_stamp: "Formatted at 2026-08-08 23:02:16 on dist-test-slave-f0tn"
server_key: "6cf91eacfdb83db43b5da52c667b0555"
server_key_iv: "8d75baef9c1c5f8a8c20fc5afb37c83a"
server_key_version: "encryptionkey@0"
I20260808 23:02:17.005813 12985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:02:17.006480 13009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:17.006632 12985 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:02:17.006718 12985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-2/wal
uuid: "46306eb2525c4adca1ff51587ee177c9"
format_stamp: "Formatted at 2026-08-08 23:02:16 on dist-test-slave-f0tn"
server_key: "6cf91eacfdb83db43b5da52c667b0555"
server_key_iv: "8d75baef9c1c5f8a8c20fc5afb37c83a"
server_key_version: "encryptionkey@0"
I20260808 23:02:17.006778 12985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:17.013084 12968 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } attempt: 2
I20260808 23:02:17.013466 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:17.013326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:38857 (local address 127.0.0.1:47308)
0808 23:02:17.013399 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:17.013437 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:38857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 23:02:17.013736 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:17.013665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:38857 (local address 127.0.0.1:47312)
0808 23:02:17.013697 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:17.013719 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:38857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260808 23:02:17.013825 12968 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.60:38857: Network error: Client connection negotiation failed: client connection to 127.23.194.60:38857: connect: Connection refused (error 111)
I20260808 23:02:17.016223 12985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:17.017820 12985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:17.017920 12985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:17.021934 12985 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.60:38857
I20260808 23:02:17.021965 13081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.60:38857 every 8 connection(s)
I20260808 23:02:17.022267 12985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-2/data/info.pb
I20260808 23:02:17.023141 13082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:02:17.024549 13082 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:02:17 dist-test-slave-f0tn krb5kdc[12740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230136, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.60@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:17.029778 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 12985
I20260808 23:02:17.029877 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786229981209642-24328-0/minicluster-data/master-2/wal/instance
I20260808 23:02:17.030062 24328 external_mini_cluster.cc:1428] Setting key 46d33486d792179e11778f064c512f7f
I20260808 23:02:17.033119 12864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:17.025158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58196 (local address 127.23.194.62:46129)
0808 23:02:17.025217 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:17.025219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:17.025410 (+ 191us) server_negotiation.cc:407] Connection header received
0808 23:02:17.025897 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:17.025898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:17.025907 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:17.025947 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:17.026499 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.026958 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.027858 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.028046 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.028091 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:17.030037 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:17.030039 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:17.030039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:17.030047 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:17.031847 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.032268 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.032269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.032270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.032331 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.032534 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.032535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.032535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.032681 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:17.032746 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:17.032947 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 23:02:17.033025 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 23:02:17.033232 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:17.025118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:46129 (local address 127.0.0.1:58196)
0808 23:02:17.025285 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:17.025323 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:17.025800 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:17.026006 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:17.026010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:17.026250 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:17.026426 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:17.026430 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.027016 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.027016 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:17.027799 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:17.027801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.028112 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.028113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:17.028224 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:17.028610 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:17.028617 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:17.029928 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:17.031921 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:17.031927 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:17.031929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:17.032207 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:17.032388 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:17.032388 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:17.032389 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:17.032480 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:17.032738 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:17.032741 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:17.032841 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 23:02:17.032917 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 23:02:17.033025 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":85,"threads_started":1}
I20260808 23:02:17.033730 13082 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:02:17 dist-test-slave-f0tn krb5kdc[12740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.60@KRBTEST.COM
Aug 08 23:02:17 dist-test-slave-f0tn krb5kdc[12740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230136, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.60@KRBTEST.COM for kudu/127.23.194.61@KRBTEST.COM
I20260808 23:02:17.038503 13087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:17.030420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47328 (local address 127.23.194.60:38857)
0808 23:02:17.030566 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:17.030570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:17.030612 (+ 42us) server_negotiation.cc:407] Connection header received
0808 23:02:17.030691 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:17.030692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:17.030702 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:17.030758 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:17.031415 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.032167 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.033072 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.033503 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.033559 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:17.035928 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:17.035935 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:17.035936 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:17.035942 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:17.037422 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.037776 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.037779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.037779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.037830 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.038012 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.038013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.038013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.038136 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:17.038267 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:17.038345 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 23:02:17.038413 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
I20260808 23:02:17.040251 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:17.034081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:38563 (local address 127.0.0.1:38438)
0808 23:02:17.034133 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:17.034143 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:17.034198 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:17.034316 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:17.034316 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:17.034444 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:17.034501 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:17.034502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.035052 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.035053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:17.035788 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:17.035792 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.036363 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.036364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:17.036418 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:17.036732 (+ 314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:17.036736 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:17.037825 (+ 1089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:17.039347 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:17.039348 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:17.039348 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:17.039533 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:17.039737 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:17.039738 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:17.039738 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:17.039798 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:17.039995 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:17.039996 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:17.040042 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 23:02:17.040114 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 23:02:17.040193 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 23:02:17.040293 12973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:17.034149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38438 (local address 127.23.194.61:38563)
0808 23:02:17.034189 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:17.034191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:17.034199 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:02:17.034255 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:17.034256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:17.034263 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:17.034299 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:17.034568 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.034988 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.035860 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.036123 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.036188 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:17.037893 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:17.037894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:17.037895 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:17.037901 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:17.039287 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.039607 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.039608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.039608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.039680 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.039858 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.039859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.039859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.039948 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:17.039989 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:17.040136 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 23:02:17.040205 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 23:02:17.040998 13082 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:02:17 dist-test-slave-f0tn krb5kdc[12740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
Aug 08 23:02:17 dist-test-slave-f0tn krb5kdc[12740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230136, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.60@KRBTEST.COM for kudu/127.23.194.60@KRBTEST.COM
I20260808 23:02:17.046659 12864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:17.039416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58212 (local address 127.23.194.62:46129)
0808 23:02:17.039453 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:17.039455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:17.039505 (+ 50us) server_negotiation.cc:407] Connection header received
0808 23:02:17.039586 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:17.039587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:17.039596 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:17.039658 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:17.040146 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.040764 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.041742 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.042022 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.042085 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:17.044205 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:17.044206 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:17.044207 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:17.044214 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:17.045617 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.045982 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.045983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.045983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.046026 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.046203 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.046204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.046204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.046298 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:17.046467 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:17.046504 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 23:02:17.046574 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 23:02:17.047617 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:17.041197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:38857 (local address 127.0.0.1:47334)
0808 23:02:17.041248 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:17.041260 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:17.041323 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:17.041467 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:17.041468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:17.041614 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:17.041648 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:17.041649 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.042381 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.042382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:17.042826 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:17.042827 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.043130 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.043131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:17.043183 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:17.043487 (+ 304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:17.043492 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:17.044568 (+ 1076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:17.046307 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:17.046308 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:17.046309 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:17.046493 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:17.046692 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:17.046693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:17.046694 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:17.046755 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:17.047248 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:17.047248 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:17.047292 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 23:02:17.047372 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 23:02:17.047472 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 23:02:17.047680 13087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:17.041228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47334 (local address 127.23.194.60:38857)
0808 23:02:17.041256 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:17.041258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:17.041307 (+ 49us) server_negotiation.cc:407] Connection header received
0808 23:02:17.041397 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:17.041398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:17.041407 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:17.041451 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:17.041719 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.042317 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.042889 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.043077 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.043125 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:17.044645 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:17.044646 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:17.044647 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:17.044653 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:17.046236 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.046566 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.046566 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.046567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.046628 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.047037 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.047038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.047038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.047171 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:17.047240 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:17.047400 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 23:02:17.047535 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":12}
I20260808 23:02:17.049103 13082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9: Bootstrap starting.
I20260808 23:02:17.050097 13082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9: Neither blocks nor log segments found. Creating new log.
I20260808 23:02:17.050381 13082 log.cc:824] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:17.051107 13082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9: No bootstrap required, opened a new log
Aug 08 23:02:17 dist-test-slave-f0tn krb5kdc[12740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.61@KRBTEST.COM
I20260808 23:02:17.053031 13082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } } peers { permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } } peers { permanent_uuid: "46306eb2525c4adca1ff51587ee177c9" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } }
I20260808 23:02:17.053188 13082 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:02:17.053237 13082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46306eb2525c4adca1ff51587ee177c9, State: Initialized, Role: FOLLOWER
I20260808 23:02:17.053364 13082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } } peers { permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } } peers { permanent_uuid: "46306eb2525c4adca1ff51587ee177c9" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } }
I20260808 23:02:17.053654 13089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } } peers { permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } } peers { permanent_uuid: "46306eb2525c4adca1ff51587ee177c9" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } } }
I20260808 23:02:17.053710 13082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:02:17.054462 12973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:17.047368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38442 (local address 127.23.194.61:38563)
0808 23:02:17.047405 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:17.047407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:17.047414 (+ 7us) server_negotiation.cc:407] Connection header received
0808 23:02:17.047435 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:17.047436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:17.047445 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:17.047502 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:17.048197 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.048842 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.049563 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.049836 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.049924 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:17.051777 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:17.051779 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:17.051779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:17.051787 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:17.053292 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.053669 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.053670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.053670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.053724 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.053891 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.053893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.053893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.054038 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:17.054244 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:17.054289 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 23:02:17.054373 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 23:02:17.053778 13089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:02:17.055881 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 23:02:17.056010 13103 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:02:17.056058 13103 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:02:17.056187 12985 master_runner.cc:186] Error getting master registration for 127.23.194.62:46129: OK, instance_id { permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" instance_seqno: 1786230136724317 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 23:02:17.059044 12985 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 23:02:17.065131 12864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:17.057470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58214 (local address 127.23.194.62:46129)
0808 23:02:17.057507 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:17.057509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:17.057879 (+ 370us) server_negotiation.cc:407] Connection header received
0808 23:02:17.057967 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:17.057968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:17.057978 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:17.058015 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:17.058721 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.059402 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.060351 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.060635 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.060672 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:17.062102 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:17.062104 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:17.062104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:17.062111 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:17.063649 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.064102 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.064103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.064104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.064167 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.064400 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.064401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.064402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.064539 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:17.064649 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:17.064971 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 23:02:17.065044 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 23:02:17.065500 12861 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } attempt: 2
I20260808 23:02:17.067210 12973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:17.058042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38454 (local address 127.23.194.61:38563)
0808 23:02:17.058142 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:17.058144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:17.058297 (+ 153us) server_negotiation.cc:407] Connection header received
0808 23:02:17.058396 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:17.058398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:17.058407 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:17.058438 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:17.059195 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.059830 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.060726 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.061216 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.061260 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:17.063336 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:17.063337 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:17.063337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:17.063344 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:17.065427 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.066398 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.066399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.066399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.066459 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.066661 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.066662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.066662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.066777 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:17.066851 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:17.067043 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 23:02:17.067113 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 23:02:17.067360 13087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:17.058104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47338 (local address 127.23.194.60:38857)
0808 23:02:17.058172 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:17.058173 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:17.059278 (+ 1105us) server_negotiation.cc:407] Connection header received
0808 23:02:17.059374 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:17.059375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:17.059384 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:17.059415 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:17.060179 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.060816 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.061814 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.062074 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.062116 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:17.063522 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:17.063524 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:17.063524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:17.063531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:17.065275 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.066484 (+ 1209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.066485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.066485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.066544 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.066773 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.066774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.066774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.066933 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:17.067000 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:17.067198 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 23:02:17.067276 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 23:02:17 dist-test-slave-f0tn krb5kdc[12740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230136, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for kudu/127.23.194.60@KRBTEST.COM
I20260808 23:02:17.072919 13119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:17.065892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47352 (local address 127.23.194.60:38857)
0808 23:02:17.066054 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:17.066055 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:17.066062 (+ 7us) server_negotiation.cc:407] Connection header received
0808 23:02:17.066078 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:17.066078 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:17.066086 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:17.066153 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:17.066434 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.067166 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.068172 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.068425 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.068473 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:17.070343 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:17.070344 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:17.070345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:17.070351 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:17.071900 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.072282 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.072283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.072283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.072336 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.072527 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.072527 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.072527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.072616 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:17.072659 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:17.072795 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 23:02:17.072851 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":103,"threads_started":1}
I20260808 23:02:17.072980 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:17.065698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:38857 (local address 127.0.0.1:47352)
0808 23:02:17.065754 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:17.065765 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:17.065805 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:17.066162 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:17.066163 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:17.066309 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:17.066345 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:17.066347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.067250 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.067251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:17.067933 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:17.067936 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.068524 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.068525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:17.068586 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:17.068906 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:17.068911 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:17.070241 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:17.071983 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:17.071984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:17.071984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:17.072207 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:17.072394 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:17.072394 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:17.072395 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:17.072459 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:17.072666 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:17.072667 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:17.072710 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 23:02:17.072782 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 23:02:17.072877 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 23:02:17.074291 12861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7: Bootstrap starting.
I20260808 23:02:17.074913 12861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7: Neither blocks nor log segments found. Creating new log.
I20260808 23:02:17.075165 12861 log.cc:824] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:17.075839 12861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7: No bootstrap required, opened a new log
I20260808 23:02:17.077227 12861 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } } peers { permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } } peers { permanent_uuid: "46306eb2525c4adca1ff51587ee177c9" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } }
I20260808 23:02:17.077342 12861 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:02:17.077404 12861 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14401145e7c44092b8581caf9a00d5c7, State: Initialized, Role: FOLLOWER
I20260808 23:02:17.077565 12861 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } } peers { permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } } peers { permanent_uuid: "46306eb2525c4adca1ff51587ee177c9" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } }
I20260808 23:02:17.077837 13122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } } peers { permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } } peers { permanent_uuid: "46306eb2525c4adca1ff51587ee177c9" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } } }
I20260808 23:02:17.077895 12861 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:02:17.077970 13122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 23:02:17.079627 13144 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:02:17.079813 13144 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:02:17.080389 12756 master_runner.cc:186] Error getting master registration for 127.23.194.61:38563: OK, instance_id { permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" instance_seqno: 1786230136872111 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 23:02:17.082868 12756 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 23:02:17.083494 13089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:02:17.083599 13089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } } peers { permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } } peers { permanent_uuid: "46306eb2525c4adca1ff51587ee177c9" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } }
I20260808 23:02:17.083828 13089 leader_election.cc:288] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14401145e7c44092b8581caf9a00d5c7 (127.23.194.62:46129), 73d1d5fa7b4c4e449ef64e9edbd5a39c (127.23.194.61:38563)
I20260808 23:02:17.084722 12817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "46306eb2525c4adca1ff51587ee177c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14401145e7c44092b8581caf9a00d5c7" is_pre_election: true
I20260808 23:02:17.084873 12817 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46306eb2525c4adca1ff51587ee177c9 in term 0.
I20260808 23:02:17.085108 13011 leader_election.cc:302] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14401145e7c44092b8581caf9a00d5c7, 46306eb2525c4adca1ff51587ee177c9; no voters:
I20260808 23:02:17.085603 13089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:02:17.085668 13089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:02:17.085690 13089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:02:17.086560 13089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } } peers { permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } } peers { permanent_uuid: "46306eb2525c4adca1ff51587ee177c9" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } }
I20260808 23:02:17.086669 13089 leader_election.cc:288] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [CANDIDATE]: Term 1 election: Requested vote from peers 14401145e7c44092b8581caf9a00d5c7 (127.23.194.62:46129), 73d1d5fa7b4c4e449ef64e9edbd5a39c (127.23.194.61:38563)
I20260808 23:02:17.087060 12817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "46306eb2525c4adca1ff51587ee177c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14401145e7c44092b8581caf9a00d5c7"
I20260808 23:02:17.087129 12817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:02:17.087929 12817 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46306eb2525c4adca1ff51587ee177c9 in term 1.
I20260808 23:02:17.088122 13011 leader_election.cc:302] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14401145e7c44092b8581caf9a00d5c7, 46306eb2525c4adca1ff51587ee177c9; no voters:
I20260808 23:02:17.088271 13089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:02:17.088436 13089 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [term 1 LEADER]: Becoming Leader. State: Replica: 46306eb2525c4adca1ff51587ee177c9, State: Running, Role: LEADER
I20260808 23:02:17.088527 13089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } } peers { permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } } peers { permanent_uuid: "46306eb2525c4adca1ff51587ee177c9" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } }
I20260808 23:02:17.089041 13146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46306eb2525c4adca1ff51587ee177c9. Latest consensus state: current_term: 1 leader_uuid: "46306eb2525c4adca1ff51587ee177c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } } peers { permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } } peers { permanent_uuid: "46306eb2525c4adca1ff51587ee177c9" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } } }
I20260808 23:02:17.089151 13146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [sys.catalog]: This master's current role is: LEADER
I20260808 23:02:17.089516 13148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:02:17.089928 13148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:02:17.092072 12817 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7 [term 1 FOLLOWER]: Refusing update from remote peer 46306eb2525c4adca1ff51587ee177c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:02:17.092376 13146 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:02:17.093761 13122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46306eb2525c4adca1ff51587ee177c9. Latest consensus state: current_term: 1 leader_uuid: "46306eb2525c4adca1ff51587ee177c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } } peers { permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } } peers { permanent_uuid: "46306eb2525c4adca1ff51587ee177c9" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } } }
I20260808 23:02:17.093840 13122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:02:17.095196 13146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "46306eb2525c4adca1ff51587ee177c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } } peers { permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } } peers { permanent_uuid: "46306eb2525c4adca1ff51587ee177c9" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } } }
I20260808 23:02:17.095269 13146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [sys.catalog]: This master's current role is: LEADER
I20260808 23:02:17.095412 13122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "46306eb2525c4adca1ff51587ee177c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } } peers { permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } } peers { permanent_uuid: "46306eb2525c4adca1ff51587ee177c9" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } } }
I20260808 23:02:17.095471 13122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:02:17.095528 13150 mvcc.cc:205] Tried to move back new op lower bound from 7316398641526833152 to 7316398641515245568. Current Snapshot: MvccSnapshot[applied={T|T < 7316398641526833152}]
I20260808 23:02:17.095809 13152 mvcc.cc:205] Tried to move back new op lower bound from 7316398641526833152 to 7316398641515245568. Current Snapshot: MvccSnapshot[applied={T|T < 7316398641526833152}]
I20260808 23:02:17.096300 13148 catalog_manager.cc:1347] Generated new cluster ID: 8152da1d000743ad9c7e14fab20584e5
I20260808 23:02:17.096347 13148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:02:17.104955 12968 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } attempt: 3
Aug 08 23:02:17 dist-test-slave-f0tn krb5kdc[12740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230136, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.61@KRBTEST.COM for kudu/127.23.194.60@KRBTEST.COM
I20260808 23:02:17.114744 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:17.105273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:38857 (local address 127.0.0.1:47364)
0808 23:02:17.105321 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:17.105331 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:17.105367 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:17.105574 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:17.105575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:17.105764 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:17.105811 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:17.105813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.107345 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.107346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:17.108157 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:17.108160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.108765 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.108766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:17.108837 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:17.109332 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:17.109337 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:17.111032 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:17.113592 (+ 2560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:17.113594 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:17.113594 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:17.113811 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:17.114041 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:17.114042 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:17.114042 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:17.114114 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:17.114422 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:17.114423 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:17.114478 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 23:02:17.114547 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 23:02:17.114643 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 23:02:17.115322 13119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:17.105321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47364 (local address 127.23.194.60:38857)
0808 23:02:17.105362 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:17.105364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:17.105400 (+ 36us) server_negotiation.cc:407] Connection header received
0808 23:02:17.105462 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:17.105464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:17.105477 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:17.105530 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:17.105889 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.106658 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.108281 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:17.108635 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:17.108695 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:17.111131 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:17.111132 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:17.111133 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:17.111140 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:17.113482 (+ 2342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.113894 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.113895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.113895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.113965 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:17.114182 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:17.114182 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:17.114183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:17.114328 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:17.114384 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:17.115151 (+ 767us) server_negotiation.cc:299] Negotiation successful
0808 23:02:17.115236 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 23:02:17.117098 12968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c: Bootstrap starting.
I20260808 23:02:17.118009 12968 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c: Neither blocks nor log segments found. Creating new log.
I20260808 23:02:17.118299 12968 log.cc:824] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:17.119896 12968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c: No bootstrap required, opened a new log
I20260808 23:02:17.121912 12968 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } } peers { permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } } peers { permanent_uuid: "46306eb2525c4adca1ff51587ee177c9" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } }
I20260808 23:02:17.122036 12968 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:02:17.122069 12968 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73d1d5fa7b4c4e449ef64e9edbd5a39c, State: Initialized, Role: FOLLOWER
I20260808 23:02:17.122184 12968 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } } peers { permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } } peers { permanent_uuid: "46306eb2525c4adca1ff51587ee177c9" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } }
I20260808 23:02:17.122473 12968 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:02:17.122525 12935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "46306eb2525c4adca1ff51587ee177c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" is_pre_election: true
I20260808 23:02:17.122673 12935 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46306eb2525c4adca1ff51587ee177c9 in term 0.
I20260808 23:02:17.122777 12934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "46306eb2525c4adca1ff51587ee177c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c"
I20260808 23:02:17.122834 12934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:02:17.123704 12934 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46306eb2525c4adca1ff51587ee177c9 in term 1.
I20260808 23:02:17.124420 12933 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c [term 1 FOLLOWER]: Refusing update from remote peer 46306eb2525c4adca1ff51587ee177c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:02:17.124761 13089 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:02:17.127419 13189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46306eb2525c4adca1ff51587ee177c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } } peers { permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } } peers { permanent_uuid: "46306eb2525c4adca1ff51587ee177c9" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } } }
I20260808 23:02:17.127521 13189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:02:17.127573 13189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46306eb2525c4adca1ff51587ee177c9. Latest consensus state: current_term: 1 leader_uuid: "46306eb2525c4adca1ff51587ee177c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } } peers { permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } } peers { permanent_uuid: "46306eb2525c4adca1ff51587ee177c9" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } } }
I20260808 23:02:17.127619 13189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:02:17.127650 13189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "46306eb2525c4adca1ff51587ee177c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } } peers { permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } } peers { permanent_uuid: "46306eb2525c4adca1ff51587ee177c9" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } } }
I20260808 23:02:17.127689 13189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:02:17.128405 13146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "46306eb2525c4adca1ff51587ee177c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14401145e7c44092b8581caf9a00d5c7" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 46129 } } peers { permanent_uuid: "73d1d5fa7b4c4e449ef64e9edbd5a39c" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 38563 } } peers { permanent_uuid: "46306eb2525c4adca1ff51587ee177c9" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 38857 } } }
I20260808 23:02:17.128499 13146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9 [sys.catalog]: This master's current role is: LEADER
I20260808 23:02:17.128664 13185 mvcc.cc:205] Tried to move back new op lower bound from 7316398641526833152 to 7316398641515245568. Current Snapshot: MvccSnapshot[applied={T|T < 7316398641526833152}]
I20260808 23:02:17.129012 13148 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:02:17.129748 13148 catalog_manager.cc:1504] Loading token signing keys...
W20260808 23:02:17.130707 13183 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:02:17.130765 13183 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:02:17.145505 13148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46306eb2525c4adca1ff51587ee177c9: Generated new TSK 0
I20260808 23:02:17.145846 13148 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:02:18.080314 13144 catalog_manager.cc:1259] Loaded cluster ID: 8152da1d000743ad9c7e14fab20584e5
I20260808 23:02:18.080394 13144 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7: loading cluster ID for follower catalog manager: success
I20260808 23:02:18.081233 13144 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7: acquiring CA information for follower catalog manager: success
I20260808 23:02:18.081593 13144 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 14401145e7c44092b8581caf9a00d5c7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 23:02:18.131192 13183 catalog_manager.cc:1259] Loaded cluster ID: 8152da1d000743ad9c7e14fab20584e5
I20260808 23:02:18.131264 13183 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c: loading cluster ID for follower catalog manager: success
I20260808 23:02:18.132066 13183 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c: acquiring CA information for follower catalog manager: success
I20260808 23:02:18.132373 13183 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 73d1d5fa7b4c4e449ef64e9edbd5a39c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 23:02:18.206456 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 12756
I20260808 23:02:18.218362 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 12868
I20260808 23:02:18.231705 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 12985
2026-08-08T23:02:18Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2900 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:02:18 dist-test-slave-f0tn krb5kdc[13210](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:02:18 dist-test-slave-f0tn krb5kdc[13210](info): set up 2 sockets
Aug 08 23:02:18 dist-test-slave-f0tn krb5kdc[13210](info): commencing operation
krb5kdc: starting...
W20260808 23:02:19.369081 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:02:19 dist-test-slave-f0tn krb5kdc[13210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:02:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:02:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:02:19.483850 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:36887
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:46007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:36887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:02:19.567731 13231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:19.567965 13231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:19.567998 13231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:19.569257 13231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:02:19.569316 13231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:19.569332 13231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:19.569360 13231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:02:19.569407 13231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:02:19.571099 13231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:46007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:36887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:36887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:19.571376 13231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:19.571619 13231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:19.576735 13239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:19.576802 13238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:19.576735 13242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:19.576762 13231 server_base.cc:1034] running on GCE node
I20260808 23:02:19.577095 13231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:19.577415 13231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:19.578578 13231 hybrid_clock.cc:648] HybridClock initialized: now 1786230139578560 us; error 33 us; skew 500 ppm
Aug 08 23:02:19 dist-test-slave-f0tn krb5kdc[13210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230139, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:19.580952 13231 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:02:19.581350 13231 webserver.cc:460] Webserver started at http://127.23.194.62:32875/ using document root <none> and password file <none>
I20260808 23:02:19.581589 13231 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:19.581635 13231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:19.581750 13231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:02:19.582716 13231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "66abfba2d66e4170bcdd2c198647ab6c"
format_stamp: "Formatted at 2026-08-08 23:02:19 on dist-test-slave-f0tn"
server_key: "d662de5200330cfa0d807a6f305fcf8d"
server_key_iv: "3e23e8d44ec79b7f51e6bba024df8c98"
server_key_version: "encryptionkey@0"
I20260808 23:02:19.583007 13231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "66abfba2d66e4170bcdd2c198647ab6c"
format_stamp: "Formatted at 2026-08-08 23:02:19 on dist-test-slave-f0tn"
server_key: "d662de5200330cfa0d807a6f305fcf8d"
server_key_iv: "3e23e8d44ec79b7f51e6bba024df8c98"
server_key_version: "encryptionkey@0"
I20260808 23:02:19.584167 13231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 23:02:19.584805 13251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:19.584959 13231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:19.585043 13231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "66abfba2d66e4170bcdd2c198647ab6c"
format_stamp: "Formatted at 2026-08-08 23:02:19 on dist-test-slave-f0tn"
server_key: "d662de5200330cfa0d807a6f305fcf8d"
server_key_iv: "3e23e8d44ec79b7f51e6bba024df8c98"
server_key_version: "encryptionkey@0"
I20260808 23:02:19.585103 13231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:19.604203 13231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:19.606055 13231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:19.606197 13231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:19.610152 13231 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:36887
I20260808 23:02:19.610201 13336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:36887 every 8 connection(s)
I20260808 23:02:19.610529 13231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:02:19.611394 13337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:02:19.613615 13337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c: Bootstrap starting.
I20260808 23:02:19.614126 13337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c: Neither blocks nor log segments found. Creating new log.
I20260808 23:02:19.614372 13337 log.cc:824] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:19.614981 13337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c: No bootstrap required, opened a new log
I20260808 23:02:19.616521 13337 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66abfba2d66e4170bcdd2c198647ab6c" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36887 } }
I20260808 23:02:19.616655 13337 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:02:19.616677 13337 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66abfba2d66e4170bcdd2c198647ab6c, State: Initialized, Role: FOLLOWER
I20260808 23:02:19.616784 13337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66abfba2d66e4170bcdd2c198647ab6c" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36887 } }
I20260808 23:02:19.616832 13337 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:02:19.616851 13337 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:02:19.616882 13337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:02:19.617357 13337 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66abfba2d66e4170bcdd2c198647ab6c" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36887 } }
I20260808 23:02:19.617486 13337 leader_election.cc:302] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66abfba2d66e4170bcdd2c198647ab6c; no voters:
I20260808 23:02:19.617622 13337 leader_election.cc:288] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:02:19.617666 13347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:02:19.617818 13347 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c [term 1 LEADER]: Becoming Leader. State: Replica: 66abfba2d66e4170bcdd2c198647ab6c, State: Running, Role: LEADER
I20260808 23:02:19.617843 13337 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:02:19.617976 13347 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66abfba2d66e4170bcdd2c198647ab6c" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36887 } }
I20260808 23:02:19.618367 13351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66abfba2d66e4170bcdd2c198647ab6c. Latest consensus state: current_term: 1 leader_uuid: "66abfba2d66e4170bcdd2c198647ab6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66abfba2d66e4170bcdd2c198647ab6c" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36887 } } }
I20260808 23:02:19.618505 13351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c [sys.catalog]: This master's current role is: LEADER
I20260808 23:02:19.618489 13348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66abfba2d66e4170bcdd2c198647ab6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66abfba2d66e4170bcdd2c198647ab6c" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 36887 } } }
I20260808 23:02:19.618649 13348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c [sys.catalog]: This master's current role is: LEADER
I20260808 23:02:19.618968 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 13231
I20260808 23:02:19.619068 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:02:19.619254 24328 external_mini_cluster.cc:1428] Setting key fc48f4782a1926d027aa50451a75e5a7
I20260808 23:02:19.619602 13363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:02:19.620146 13363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:02:19.621798 13363 catalog_manager.cc:1347] Generated new cluster ID: aacec6f1fc85440fb087c9ca229daa41
I20260808 23:02:19.621845 13363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 23:02:19 dist-test-slave-f0tn krb5kdc[13210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:19.628226 13363 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:02:19.628700 13363 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:02:19.632232 13373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:19.619713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47130 (local address 127.23.194.62:36887)
0808 23:02:19.619918 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:19.619928 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:19.619962 (+ 34us) server_negotiation.cc:407] Connection header received
0808 23:02:19.620064 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:19.620070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:19.620151 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:19.620671 (+ 520us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:19.621344 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:19.622747 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:19.623840 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:19.624476 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:19.624584 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:19.627381 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:19.627386 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:19.627386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:19.627400 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:19.630657 (+ 3257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:19.631057 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:19.631060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:19.631060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:19.631178 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:19.631368 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:19.631369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:19.631369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:19.631614 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:19.631753 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:19.631853 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 23:02:19.631994 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260808 23:02:19.633213 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:02:19.634955 13363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66abfba2d66e4170bcdd2c198647ab6c: Generated new TSK 0
I20260808 23:02:19.635126 13363 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:02:19.687175 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 13231
2026-08-08T23:02:19Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1457 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:02:19 dist-test-slave-f0tn krb5kdc[13389](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:02:19 dist-test-slave-f0tn krb5kdc[13389](info): set up 2 sockets
Aug 08 23:02:19 dist-test-slave-f0tn krb5kdc[13389](info): commencing operation
krb5kdc: starting...
W20260808 23:02:20.822868 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:02:20 dist-test-slave-f0tn krb5kdc[13389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:02:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:02:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:02:20.939288 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:34697
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:34697
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:02:21.019377 13410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:21.019577 13410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:21.019608 13410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:21.020737 13410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:02:21.020777 13410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:21.020797 13410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:21.020812 13410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:02:21.020828 13410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:02:21.022361 13410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:34697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:34697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:21.022617 13410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:21.022825 13410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:21.027513 13423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:21.027544 13421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:21.027518 13417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:21.027786 13410 server_base.cc:1034] running on GCE node
I20260808 23:02:21.027997 13410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:21.028275 13410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:21.029433 13410 hybrid_clock.cc:648] HybridClock initialized: now 1786230141029417 us; error 29 us; skew 500 ppm
Aug 08 23:02:21 dist-test-slave-f0tn krb5kdc[13389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230141, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:21.031822 13410 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:02:21.032261 13410 webserver.cc:460] Webserver started at http://127.23.194.62:32807/ using document root <none> and password file <none>
I20260808 23:02:21.032490 13410 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:21.032537 13410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:21.032672 13410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:02:21.033735 13410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "4041bcd342cc4f78a338ed497066a169"
format_stamp: "Formatted at 2026-08-08 23:02:21 on dist-test-slave-f0tn"
server_key: "64a98a99920b48b623070e54e1c7fb6c"
server_key_iv: "703acac7db7aa1361d95b12fb6431746"
server_key_version: "encryptionkey@0"
I20260808 23:02:21.034058 13410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "4041bcd342cc4f78a338ed497066a169"
format_stamp: "Formatted at 2026-08-08 23:02:21 on dist-test-slave-f0tn"
server_key: "64a98a99920b48b623070e54e1c7fb6c"
server_key_iv: "703acac7db7aa1361d95b12fb6431746"
server_key_version: "encryptionkey@0"
I20260808 23:02:21.035267 13410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:02:21.035972 13433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:21.036129 13410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:02:21.036203 13410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "4041bcd342cc4f78a338ed497066a169"
format_stamp: "Formatted at 2026-08-08 23:02:21 on dist-test-slave-f0tn"
server_key: "64a98a99920b48b623070e54e1c7fb6c"
server_key_iv: "703acac7db7aa1361d95b12fb6431746"
server_key_version: "encryptionkey@0"
I20260808 23:02:21.036270 13410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:21.045094 13410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:21.046818 13410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:21.046960 13410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:21.051146 13410 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:34697
I20260808 23:02:21.051177 13517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:34697 every 8 connection(s)
I20260808 23:02:21.051510 13410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:02:21.052456 13518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:02:21.053524 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 13410
I20260808 23:02:21.053656 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:02:21.053855 24328 external_mini_cluster.cc:1428] Setting key 4e83a0b3b821629c092d247ecbedd146
I20260808 23:02:21.055035 13518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169: Bootstrap starting.
I20260808 23:02:21.055744 13518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169: Neither blocks nor log segments found. Creating new log.
I20260808 23:02:21.056021 13518 log.cc:824] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:21.056702 13518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169: No bootstrap required, opened a new log
I20260808 23:02:21.058897 13518 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4041bcd342cc4f78a338ed497066a169" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34697 } }
I20260808 23:02:21.059077 13518 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:02:21.059126 13518 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4041bcd342cc4f78a338ed497066a169, State: Initialized, Role: FOLLOWER
I20260808 23:02:21.059264 13518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4041bcd342cc4f78a338ed497066a169" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34697 } }
I20260808 23:02:21.059360 13518 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:02:21.059398 13518 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:02:21.059448 13518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:02:21.060110 13518 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4041bcd342cc4f78a338ed497066a169" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34697 } }
I20260808 23:02:21.060209 13518 leader_election.cc:302] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4041bcd342cc4f78a338ed497066a169; no voters:
I20260808 23:02:21.060359 13518 leader_election.cc:288] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:02:21.060395 13525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:02:21.060554 13525 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 1 LEADER]: Becoming Leader. State: Replica: 4041bcd342cc4f78a338ed497066a169, State: Running, Role: LEADER
Aug 08 23:02:21 dist-test-slave-f0tn krb5kdc[13389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:21.060626 13518 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:02:21.060699 13525 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4041bcd342cc4f78a338ed497066a169" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34697 } }
I20260808 23:02:21.061008 13525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4041bcd342cc4f78a338ed497066a169. Latest consensus state: current_term: 1 leader_uuid: "4041bcd342cc4f78a338ed497066a169" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4041bcd342cc4f78a338ed497066a169" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34697 } } }
I20260808 23:02:21.061066 13525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [sys.catalog]: This master's current role is: LEADER
I20260808 23:02:21.061554 13535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:02:21.061841 13529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4041bcd342cc4f78a338ed497066a169" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4041bcd342cc4f78a338ed497066a169" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34697 } } }
I20260808 23:02:21.061910 13529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [sys.catalog]: This master's current role is: LEADER
I20260808 23:02:21.062114 13535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:02:21.064105 13535 catalog_manager.cc:1347] Generated new cluster ID: fa40ffa10f944ec68032d3520d6b0030
I20260808 23:02:21.064167 13535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:02:21.065047 13523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:21.054364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58972 (local address 127.23.194.62:34697)
0808 23:02:21.054528 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:21.054536 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:21.054560 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:02:21.054945 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:21.054951 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:21.054993 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:21.055115 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:21.055846 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.056953 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.058033 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.058418 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.058491 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:21.061402 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:21.061406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:21.061407 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:21.061419 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:21.063507 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:21.063891 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:21.063894 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:21.063894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:21.064003 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:21.064237 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:21.064238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:21.064238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:21.064464 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:21.064609 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:21.064712 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 23:02:21.064842 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
I20260808 23:02:21.082160 13535 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:02:21.082891 13535 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:02:21.088683 13535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169: Generated new TSK 0
I20260808 23:02:21.088867 13535 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
I20260808 23:02:21.115487 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34697
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:02:21.195253 13563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:21.195444 13563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:21.195466 13563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:21.196578 13563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:21.196620 13563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:21.196648 13563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:02:21.198094 13563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:21.198303 13563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:21.198508 13563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 23:02:21.203409 13563 server_base.cc:1034] running on GCE node
W20260808 23:02:21.203326 13577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:21.203357 13573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:21.203343 13574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:21.203750 13563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:21.203992 13563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:21.205127 13563 hybrid_clock.cc:648] HybridClock initialized: now 1786230141205113 us; error 28 us; skew 500 ppm
Aug 08 23:02:21 dist-test-slave-f0tn krb5kdc[13389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230141, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:21.207453 13563 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 23:02:21.207847 13563 webserver.cc:460] Webserver started at http://127.23.194.1:46781/ using document root <none> and password file <none>
I20260808 23:02:21.208038 13563 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:21.208086 13563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:21.208210 13563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:02:21.209182 13563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "1228fa7b2dab4969ad74438a5495e98f"
format_stamp: "Formatted at 2026-08-08 23:02:21 on dist-test-slave-f0tn"
server_key: "7d2867380c386527f52f56945fb81d5c"
server_key_iv: "c0b226ee360cecf9a6827bc4189b71e0"
server_key_version: "encryptionkey@0"
I20260808 23:02:21.209532 13563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "1228fa7b2dab4969ad74438a5495e98f"
format_stamp: "Formatted at 2026-08-08 23:02:21 on dist-test-slave-f0tn"
server_key: "7d2867380c386527f52f56945fb81d5c"
server_key_iv: "c0b226ee360cecf9a6827bc4189b71e0"
server_key_version: "encryptionkey@0"
I20260808 23:02:21.210727 13563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 23:02:21.211383 13586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:21.211530 13563 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:02:21.211613 13563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "1228fa7b2dab4969ad74438a5495e98f"
format_stamp: "Formatted at 2026-08-08 23:02:21 on dist-test-slave-f0tn"
server_key: "7d2867380c386527f52f56945fb81d5c"
server_key_iv: "c0b226ee360cecf9a6827bc4189b71e0"
server_key_version: "encryptionkey@0"
I20260808 23:02:21.211685 13563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:21.220667 13563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:21.222375 13563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:21.222506 13563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:21.222759 13563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:02:21.223054 13563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:02:21.223088 13563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:21.223117 13563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:02:21.223135 13563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:21.228618 13563 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:34835
I20260808 23:02:21.228686 13759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:34835 every 8 connection(s)
I20260808 23:02:21.228955 13563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:02:21.230064 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 13563
I20260808 23:02:21.230186 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:02:21.230376 24328 external_mini_cluster.cc:1428] Setting key 57024d1226124f0ddf057cbe75923776
Aug 08 23:02:21 dist-test-slave-f0tn krb5kdc[13389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230141, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:21.237919 13523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:21.229665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:33843 (local address 127.23.194.62:34697)
0808 23:02:21.229717 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:21.229719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:21.229896 (+ 177us) server_negotiation.cc:407] Connection header received
0808 23:02:21.230454 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:21.230455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:21.230468 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:21.230521 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:21.231502 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.232000 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.233017 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.233224 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.233279 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:21.235362 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:21.235364 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:21.235364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:21.235372 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:21.236629 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:21.236992 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:21.236993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:21.236993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:21.237046 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:21.237274 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:21.237275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:21.237275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:21.237418 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:21.237475 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:21.237736 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 23:02:21.237827 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 23:02:21.238123 13762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:21.229629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.1:33843)
0808 23:02:21.229783 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:21.229820 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:21.230359 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:21.230589 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:21.230593 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:21.231220 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:21.231407 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:21.231412 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.232089 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.232091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:21.232934 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:21.232938 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.233299 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.233300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:21.233455 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:21.233930 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:21.233939 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:21.235275 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:21.236712 (+ 1437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:21.236719 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:21.236721 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:21.236922 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:21.237103 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:21.237103 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:21.237104 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:21.237215 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:21.237477 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:21.237484 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:21.237614 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 23:02:21.237703 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 23:02:21.237851 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for kudu/127.23.194.2@KRBTEST.COM; defaulting to no policy
I20260808 23:02:21.238571 13760 heartbeater.cc:347] Connected to a master server at 127.23.194.62:34697
I20260808 23:02:21.238663 13760 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:21.238857 13760 heartbeater.cc:511] Master 127.23.194.62:34697 requested a full tablet report, sending...
I20260808 23:02:21.239387 13465 ts_manager.cc:194] Registered new tserver with Master: 1228fa7b2dab4969ad74438a5495e98f (127.23.194.1:34835)
I20260808 23:02:21.240056 13465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:33843
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
I20260808 23:02:21.254172 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34697
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:02:21.334862 13769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:21.335073 13769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:21.335106 13769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:21.336239 13769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:21.336280 13769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:21.336314 13769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:02:21.337783 13769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:21.338017 13769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:21.338263 13769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:21.343420 13780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:21.343415 13776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:21.343415 13782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:21.343431 13769 server_base.cc:1034] running on GCE node
I20260808 23:02:21.343791 13769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:21.344045 13769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:21.345214 13769 hybrid_clock.cc:648] HybridClock initialized: now 1786230141345193 us; error 31 us; skew 500 ppm
Aug 08 23:02:21 dist-test-slave-f0tn krb5kdc[13389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230141, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:21.347872 13769 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 23:02:21.348330 13769 webserver.cc:460] Webserver started at http://127.23.194.2:43405/ using document root <none> and password file <none>
I20260808 23:02:21.348567 13769 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:21.348616 13769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:21.348757 13769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:02:21.349858 13769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "686c6bae2d95420bb3911980f6dde11d"
format_stamp: "Formatted at 2026-08-08 23:02:21 on dist-test-slave-f0tn"
server_key: "ee7aae15ef2034ac70b918f02ebd95a3"
server_key_iv: "49ecd2815841439a70b89362cfc2f75d"
server_key_version: "encryptionkey@0"
I20260808 23:02:21.350204 13769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "686c6bae2d95420bb3911980f6dde11d"
format_stamp: "Formatted at 2026-08-08 23:02:21 on dist-test-slave-f0tn"
server_key: "ee7aae15ef2034ac70b918f02ebd95a3"
server_key_iv: "49ecd2815841439a70b89362cfc2f75d"
server_key_version: "encryptionkey@0"
I20260808 23:02:21.351454 13769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:02:21.352151 13798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:21.352304 13769 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:02:21.352392 13769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "686c6bae2d95420bb3911980f6dde11d"
format_stamp: "Formatted at 2026-08-08 23:02:21 on dist-test-slave-f0tn"
server_key: "ee7aae15ef2034ac70b918f02ebd95a3"
server_key_iv: "49ecd2815841439a70b89362cfc2f75d"
server_key_version: "encryptionkey@0"
I20260808 23:02:21.352455 13769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:21.367563 13769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:21.369728 13769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:21.369876 13769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:21.370126 13769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:02:21.370438 13769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:02:21.370473 13769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:21.370503 13769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:02:21.370522 13769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:21.376713 13769 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:43531
I20260808 23:02:21.376752 13968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:43531 every 8 connection(s)
I20260808 23:02:21.377064 13769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:02:21.378911 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 13769
I20260808 23:02:21.379010 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:02:21.379194 24328 external_mini_cluster.cc:1428] Setting key c450843fc50a1e865a9332da0497bf89
Aug 08 23:02:21 dist-test-slave-f0tn krb5kdc[13389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230141, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:21.386698 13523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:21.377847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:38227 (local address 127.23.194.62:34697)
0808 23:02:21.377908 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:21.377911 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:21.378109 (+ 198us) server_negotiation.cc:407] Connection header received
0808 23:02:21.378573 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:21.378574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:21.378583 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:21.378638 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:21.379647 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.380238 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.381561 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.381794 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.381856 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:21.383928 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:21.383930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:21.383930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:21.383938 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:21.385437 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:21.385813 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:21.385814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:21.385815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:21.385877 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:21.386089 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:21.386090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:21.386090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:21.386239 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:21.386310 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:21.386526 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 23:02:21.386608 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 23:02:21.386768 13973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:21.377817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.2:38227)
0808 23:02:21.377979 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:21.378022 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:21.378499 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:21.378693 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:21.378697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:21.379316 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:21.379555 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:21.379562 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.380335 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.380336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:21.381492 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:21.381495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.381883 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.381884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:21.382006 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:21.382491 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:21.382498 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:21.383836 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:21.385513 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:21.385519 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:21.385521 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:21.385739 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:21.385939 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:21.385939 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:21.385940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:21.386030 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:21.386298 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:21.386301 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:21.386409 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 23:02:21.386485 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 23:02:21.386599 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260808 23:02:21.387256 13970 heartbeater.cc:347] Connected to a master server at 127.23.194.62:34697
I20260808 23:02:21.387351 13970 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:21.387526 13970 heartbeater.cc:511] Master 127.23.194.62:34697 requested a full tablet report, sending...
I20260808 23:02:21.387938 13465 ts_manager.cc:194] Registered new tserver with Master: 686c6bae2d95420bb3911980f6dde11d (127.23.194.2:43531)
I20260808 23:02:21.388485 13465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:38227
WARNING: no policy specified for kudu/127.23.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
I20260808 23:02:21.405438 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34697
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:02:21.486624 13979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:21.486812 13979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:21.486833 13979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:21.488008 13979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:21.488049 13979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:21.488075 13979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:02:21.489580 13979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:21.489813 13979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:21.490023 13979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:21.494733 13990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:21.494881 13979 server_base.cc:1034] running on GCE node
W20260808 23:02:21.494846 13988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:21.494738 13986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:21.495177 13979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:21.495400 13979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:21.496538 13979 hybrid_clock.cc:648] HybridClock initialized: now 1786230141496529 us; error 26 us; skew 500 ppm
Aug 08 23:02:21 dist-test-slave-f0tn krb5kdc[13389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230141, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:21.498843 13979 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 23:02:21.499279 13979 webserver.cc:460] Webserver started at http://127.23.194.3:44753/ using document root <none> and password file <none>
I20260808 23:02:21.499485 13979 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:21.499531 13979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:21.499658 13979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:02:21.500741 13979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "95875a4556fa43c5b39f640233a9fe13"
format_stamp: "Formatted at 2026-08-08 23:02:21 on dist-test-slave-f0tn"
server_key: "d88f02659256f77b77348ffabc26534a"
server_key_iv: "a80481bd6259776befd898c9fd4a0e63"
server_key_version: "encryptionkey@0"
I20260808 23:02:21.501145 13979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "95875a4556fa43c5b39f640233a9fe13"
format_stamp: "Formatted at 2026-08-08 23:02:21 on dist-test-slave-f0tn"
server_key: "d88f02659256f77b77348ffabc26534a"
server_key_iv: "a80481bd6259776befd898c9fd4a0e63"
server_key_version: "encryptionkey@0"
I20260808 23:02:21.502894 13979 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:02:21.503764 14005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:21.503914 13979 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:02:21.504004 13979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "95875a4556fa43c5b39f640233a9fe13"
format_stamp: "Formatted at 2026-08-08 23:02:21 on dist-test-slave-f0tn"
server_key: "d88f02659256f77b77348ffabc26534a"
server_key_iv: "a80481bd6259776befd898c9fd4a0e63"
server_key_version: "encryptionkey@0"
I20260808 23:02:21.504072 13979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:21.523993 13979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:21.525610 13979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:21.525746 13979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:21.525945 13979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:02:21.526232 13979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:02:21.526266 13979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:21.526294 13979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:02:21.526311 13979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:21.532119 13979 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:43727
I20260808 23:02:21.532191 14181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:43727 every 8 connection(s)
I20260808 23:02:21.532459 13979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
Aug 08 23:02:21 dist-test-slave-f0tn krb5kdc[13389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230141, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:21.540485 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 13979
I20260808 23:02:21.540575 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:02:21.540755 24328 external_mini_cluster.cc:1428] Setting key f2a5284fb87cdd515d1ea5d0960c7960
I20260808 23:02:21.541404 13523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:21.533109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:41623 (local address 127.23.194.62:34697)
0808 23:02:21.533190 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:21.533192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:21.533350 (+ 158us) server_negotiation.cc:407] Connection header received
0808 23:02:21.533905 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:21.533906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:21.533920 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:21.533969 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:21.534887 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.535387 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.536415 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.536602 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.536658 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:21.538733 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:21.538735 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:21.538735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:21.538743 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:21.540050 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:21.540437 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:21.540438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:21.540438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:21.540495 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:21.540773 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:21.540774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:21.540774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:21.540898 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:21.540950 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:21.541230 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 23:02:21.541312 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 23:02:21.541601 14185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:21.533052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.3:41623)
0808 23:02:21.533225 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:21.533268 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:21.533797 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:21.534048 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:21.534052 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:21.534598 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:21.534792 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:21.534797 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.535475 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.535476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:21.536332 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:21.536335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.536680 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.536682 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:21.536797 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:21.537261 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:21.537270 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:21.538647 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:21.540127 (+ 1480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:21.540133 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:21.540135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:21.540368 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:21.540564 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:21.540565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:21.540565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:21.540704 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:21.540949 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:21.540956 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:21.541098 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 23:02:21.541208 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 23:02:21.541362 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":82,"threads_started":1}
I20260808 23:02:21.542179 14183 heartbeater.cc:347] Connected to a master server at 127.23.194.62:34697
I20260808 23:02:21.542258 14183 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:21.542397 14183 heartbeater.cc:511] Master 127.23.194.62:34697 requested a full tablet report, sending...
I20260808 23:02:21.542688 13465 ts_manager.cc:194] Registered new tserver with Master: 95875a4556fa43c5b39f640233a9fe13 (127.23.194.3:43727)
I20260808 23:02:21.543045 13465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:41623
I20260808 23:02:21.552556 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:02:21.559790 13523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:21.553973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58984 (local address 127.23.194.62:34697)
0808 23:02:21.554058 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:21.554060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:21.554130 (+ 70us) server_negotiation.cc:407] Connection header received
0808 23:02:21.554204 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:21.554205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:21.554215 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:21.554258 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:21.554824 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.555362 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.556069 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.556281 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.556335 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:21.557466 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:21.557468 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:21.557468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:21.557475 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:21.558823 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:21.559171 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:21.559172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:21.559172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:21.559234 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:21.559381 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:21.559382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:21.559382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:21.559516 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:21.559590 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:21.559662 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 23:02:21.559723 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 23:02:21.561270 13465 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:02:21.563081 13465 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:02:21.568950 14199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:21.565106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:34835 (local address 127.0.0.1:35630)
0808 23:02:21.565420 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:21.565431 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:21.565488 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:21.565677 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:21.565681 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:21.565683 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:02:21.565751 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:21.565752 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.566795 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.566796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:21.567766 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:21.567768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.568650 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.568651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:21.568739 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:21.568753 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 23:02:21.568823 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 23:02:21.568875 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":2,"thread_start_us":53,"threads_started":1}
I20260808 23:02:21.569211 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:21.565299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35630 (local address 127.23.194.1:34835)
0808 23:02:21.565517 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:21.565522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:21.565532 (+ 10us) server_negotiation.cc:407] Connection header received
0808 23:02:21.565572 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:21.565574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:21.565589 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:21.565671 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:02:21.565833 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.566695 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.567874 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.568556 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.568627 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:21.568692 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:21.568856 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 23:02:21.568901 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":102,"threads_started":1}
I20260808 23:02:21.569278 14196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:21.565106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:43531 (local address 127.0.0.1:59924)
0808 23:02:21.565261 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:21.565299 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:21.565355 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:21.566084 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:21.566085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:21.566087 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:02:21.566122 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:21.566123 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.567384 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.567385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:21.568229 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:21.568231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.569068 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.569069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:21.569130 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:21.569147 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 23:02:21.569174 (+ 27us) client_negotiation.cc:240] Negotiation successful
0808 23:02:21.569213 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":1,"thread_start_us":52,"threads_started":1}
I20260808 23:02:21.569592 14198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:21.565206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59924 (local address 127.23.194.2:43531)
0808 23:02:21.565826 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:21.565830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:21.565840 (+ 10us) server_negotiation.cc:407] Connection header received
0808 23:02:21.565877 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:21.565879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:21.565964 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:21.566037 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:02:21.566218 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.567269 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.568303 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.568968 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.569369 (+ 401us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:21.569457 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:21.569482 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 23:02:21.569520 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":83,"threads_started":1}
I20260808 23:02:21.570004 13662 tablet_service.cc:1467] Processing CreateTablet for tablet e720a00bd9b5404aa7bffd21d7000256 (DEFAULT_TABLE table=test-table [id=dcf2fd4c734b4f418625ee905c73c66f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:02:21.570284 14197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:21.565293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:43727 (local address 127.0.0.1:50578)
0808 23:02:21.565441 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:21.565450 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:21.565491 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:21.565849 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:21.565850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:21.565852 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:02:21.565903 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:21.565905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.568297 (+ 2392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.568298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:21.569122 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:21.569125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.570045 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.570046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:21.570108 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:21.570112 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:02:21.570181 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 23:02:21.570220 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":67,"threads_started":1}
I20260808 23:02:21.570257 13865 tablet_service.cc:1467] Processing CreateTablet for tablet e720a00bd9b5404aa7bffd21d7000256 (DEFAULT_TABLE table=test-table [id=dcf2fd4c734b4f418625ee905c73c66f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:02:21.570386 13662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e720a00bd9b5404aa7bffd21d7000256. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:02:21.570566 13865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e720a00bd9b5404aa7bffd21d7000256. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:02:21.570582 14204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:21.565512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50578 (local address 127.23.194.3:43727)
0808 23:02:21.565704 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:21.565709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:21.565719 (+ 10us) server_negotiation.cc:407] Connection header received
0808 23:02:21.565751 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:21.565753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:21.565766 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:21.565834 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:02:21.566043 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.567284 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.569196 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.569970 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.570037 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:21.570094 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:21.570250 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 23:02:21.570297 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":92,"threads_started":1}
I20260808 23:02:21.571135 14077 tablet_service.cc:1467] Processing CreateTablet for tablet e720a00bd9b5404aa7bffd21d7000256 (DEFAULT_TABLE table=test-table [id=dcf2fd4c734b4f418625ee905c73c66f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:02:21.571421 14077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e720a00bd9b5404aa7bffd21d7000256. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:02:21.573729 14206 tablet_bootstrap.cc:492] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f: Bootstrap starting.
I20260808 23:02:21.573962 14205 tablet_bootstrap.cc:492] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d: Bootstrap starting.
I20260808 23:02:21.574045 14208 tablet_bootstrap.cc:492] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13: Bootstrap starting.
I20260808 23:02:21.574651 14208 tablet_bootstrap.cc:654] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13: Neither blocks nor log segments found. Creating new log.
I20260808 23:02:21.574651 14205 tablet_bootstrap.cc:654] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d: Neither blocks nor log segments found. Creating new log.
I20260808 23:02:21.575227 14208 log.cc:824] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:21.575227 14205 log.cc:824] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:21.575969 14205 tablet_bootstrap.cc:492] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d: No bootstrap required, opened a new log
I20260808 23:02:21.575975 14206 tablet_bootstrap.cc:654] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f: Neither blocks nor log segments found. Creating new log.
I20260808 23:02:21.576031 14205 ts_tablet_manager.cc:1397] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 23:02:21.576212 14208 tablet_bootstrap.cc:492] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13: No bootstrap required, opened a new log
I20260808 23:02:21.576290 14208 ts_tablet_manager.cc:1397] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:02:21.576563 14206 log.cc:824] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:21.577329 14206 tablet_bootstrap.cc:492] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f: No bootstrap required, opened a new log
I20260808 23:02:21.577425 14206 ts_tablet_manager.cc:1397] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 23:02:21.577925 14208 raft_consensus.cc:348] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } }
I20260808 23:02:21.577899 14205 raft_consensus.cc:348] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } }
I20260808 23:02:21.578053 14208 raft_consensus.cc:374] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:02:21.578060 14205 raft_consensus.cc:374] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:02:21.578086 14208 raft_consensus.cc:729] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95875a4556fa43c5b39f640233a9fe13, State: Initialized, Role: FOLLOWER
I20260808 23:02:21.578094 14205 raft_consensus.cc:729] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 686c6bae2d95420bb3911980f6dde11d, State: Initialized, Role: FOLLOWER
I20260808 23:02:21.578222 14205 consensus_queue.cc:260] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } }
I20260808 23:02:21.578243 14208 consensus_queue.cc:260] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } }
I20260808 23:02:21.578508 14205 ts_tablet_manager.cc:1428] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 23:02:21.578514 14208 ts_tablet_manager.cc:1428] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 23:02:21.578691 14183 heartbeater.cc:503] Master 127.23.194.62:34697 was elected leader, sending a full tablet report...
I20260808 23:02:21.578717 13970 heartbeater.cc:503] Master 127.23.194.62:34697 was elected leader, sending a full tablet report...
I20260808 23:02:21.579362 14206 raft_consensus.cc:348] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } }
I20260808 23:02:21.579525 14206 raft_consensus.cc:374] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:02:21.579571 14206 raft_consensus.cc:729] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1228fa7b2dab4969ad74438a5495e98f, State: Initialized, Role: FOLLOWER
I20260808 23:02:21.579690 14206 consensus_queue.cc:260] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } }
I20260808 23:02:21.580016 14206 ts_tablet_manager.cc:1428] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260808 23:02:21.580027 13760 heartbeater.cc:503] Master 127.23.194.62:34697 was elected leader, sending a full tablet report...
W20260808 23:02:21.627693 13971 tablet.cc:2369] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:02:21.692705 14213 raft_consensus.cc:482] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:02:21.692840 14213 raft_consensus.cc:504] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } }
I20260808 23:02:21.693236 14213 leader_election.cc:288] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1228fa7b2dab4969ad74438a5495e98f (127.23.194.1:34835), 95875a4556fa43c5b39f640233a9fe13 (127.23.194.3:43727)
I20260808 23:02:21.696031 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:21.693561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:41985 (local address 127.23.194.1:34835)
0808 23:02:21.693639 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:21.693642 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:21.693654 (+ 12us) server_negotiation.cc:407] Connection header received
0808 23:02:21.693687 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:21.693688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:21.693699 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:21.693756 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:02:21.693925 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.694411 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.695333 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.695710 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.695773 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:21.695783 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:21.695929 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 23:02:21.695969 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 23:02:21.696029 13973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:21.693485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:34835 (local address 127.23.194.2:41985)
0808 23:02:21.693551 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:21.693565 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:21.693627 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:21.693781 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:21.693782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:21.693784 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:02:21.693853 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:21.693854 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.694488 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.694489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:21.695235 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:21.695237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.695783 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.695785 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:21.695860 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:21.695863 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:02:21.695909 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 23:02:21.695953 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 23:02:21.696272 14204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:21.693704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:37601 (local address 127.23.194.3:43727)
0808 23:02:21.693753 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:21.693755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:21.693844 (+ 89us) server_negotiation.cc:407] Connection header received
0808 23:02:21.693899 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:21.693900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:21.693910 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:21.693956 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:02:21.694136 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.694606 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.695520 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.695895 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.695956 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:21.695964 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:21.696163 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 23:02:21.696209 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 23:02:21.696415 13691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e720a00bd9b5404aa7bffd21d7000256" candidate_uuid: "686c6bae2d95420bb3911980f6dde11d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1228fa7b2dab4969ad74438a5495e98f" is_pre_election: true
I20260808 23:02:21.696507 14216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:21.693623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:43727 (local address 127.23.194.2:37601)
0808 23:02:21.693790 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:21.693799 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:21.693848 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:21.693979 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:21.693980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:21.693982 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:02:21.694065 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:21.694067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.694688 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.694689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:21.695419 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:21.695421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.695984 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.695985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:21.696067 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:21.696071 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:02:21.696391 (+ 320us) client_negotiation.cc:240] Negotiation successful
0808 23:02:21.696433 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":101,"threads_started":1}
I20260808 23:02:21.696597 13691 raft_consensus.cc:2454] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 686c6bae2d95420bb3911980f6dde11d in term 0.
I20260808 23:02:21.696838 13805 leader_election.cc:302] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1228fa7b2dab4969ad74438a5495e98f, 686c6bae2d95420bb3911980f6dde11d; no voters:
I20260808 23:02:21.696858 14109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e720a00bd9b5404aa7bffd21d7000256" candidate_uuid: "686c6bae2d95420bb3911980f6dde11d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95875a4556fa43c5b39f640233a9fe13" is_pre_election: true
I20260808 23:02:21.696969 14213 raft_consensus.cc:2781] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:02:21.697027 14213 raft_consensus.cc:482] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:02:21.697026 14109 raft_consensus.cc:2454] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 686c6bae2d95420bb3911980f6dde11d in term 0.
I20260808 23:02:21.697054 14213 raft_consensus.cc:3037] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:02:21.697820 14213 raft_consensus.cc:504] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } }
I20260808 23:02:21.697932 14213 leader_election.cc:288] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [CANDIDATE]: Term 1 election: Requested vote from peers 1228fa7b2dab4969ad74438a5495e98f (127.23.194.1:34835), 95875a4556fa43c5b39f640233a9fe13 (127.23.194.3:43727)
I20260808 23:02:21.698129 14109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e720a00bd9b5404aa7bffd21d7000256" candidate_uuid: "686c6bae2d95420bb3911980f6dde11d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95875a4556fa43c5b39f640233a9fe13"
I20260808 23:02:21.698125 13691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e720a00bd9b5404aa7bffd21d7000256" candidate_uuid: "686c6bae2d95420bb3911980f6dde11d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1228fa7b2dab4969ad74438a5495e98f"
I20260808 23:02:21.698200 14109 raft_consensus.cc:3037] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:02:21.698210 13691 raft_consensus.cc:3037] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:02:21.698890 14109 raft_consensus.cc:2454] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 686c6bae2d95420bb3911980f6dde11d in term 1.
I20260808 23:02:21.699026 13691 raft_consensus.cc:2454] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 686c6bae2d95420bb3911980f6dde11d in term 1.
I20260808 23:02:21.699056 13805 leader_election.cc:302] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 686c6bae2d95420bb3911980f6dde11d, 95875a4556fa43c5b39f640233a9fe13; no voters:
I20260808 23:02:21.699196 14213 raft_consensus.cc:2781] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:02:21.699369 14213 raft_consensus.cc:686] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [term 1 LEADER]: Becoming Leader. State: Replica: 686c6bae2d95420bb3911980f6dde11d, State: Running, Role: LEADER
I20260808 23:02:21.699467 14213 consensus_queue.cc:237] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } }
I20260808 23:02:21.700383 13465 catalog_manager.cc:5495] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d reported cstate change: term changed from 0 to 1, leader changed from <none> to 686c6bae2d95420bb3911980f6dde11d (127.23.194.2). New cstate: current_term: 1 leader_uuid: "686c6bae2d95420bb3911980f6dde11d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } health_report { overall_health: UNKNOWN } } }
I20260808 23:02:21.719477 14198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:21.717141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59928 (local address 127.23.194.2:43531)
0808 23:02:21.717200 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:21.717202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:21.717211 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:02:21.717252 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:21.717252 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:21.717269 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:21.717328 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:02:21.717472 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.717937 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.718739 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:21.718929 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:21.718993 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:21.719173 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:02:21.719273 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:02:21.719330 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:21.719361 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 23:02:21.719377 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 23:02:21.721230 13691 raft_consensus.cc:1261] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Refusing update from remote peer 686c6bae2d95420bb3911980f6dde11d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:02:21.721230 14109 raft_consensus.cc:1261] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13 [term 1 FOLLOWER]: Refusing update from remote peer 686c6bae2d95420bb3911980f6dde11d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:02:21.721618 14217 consensus_queue.cc:1035] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [LEADER]: Connected to new peer: Peer: permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:02:21.721776 14213 consensus_queue.cc:1035] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:02:21.724164 14222 mvcc.cc:205] Tried to move back new op lower bound from 7316398660487446528 to 7316398660401627136. Current Snapshot: MvccSnapshot[applied={T|T < 7316398660487446528}]
I20260808 23:02:21.726578 13463 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:02:21.726658 13463 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:02:21.727370 13463 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169: Sending DeleteTablet for 3 replicas of tablet e720a00bd9b5404aa7bffd21d7000256
I20260808 23:02:21.727716 14077 tablet_service.cc:1514] Processing DeleteTablet for tablet e720a00bd9b5404aa7bffd21d7000256 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:02:21 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:50578
I20260808 23:02:21.727728 13662 tablet_service.cc:1514] Processing DeleteTablet for tablet e720a00bd9b5404aa7bffd21d7000256 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:02:21 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:35630
I20260808 23:02:21.727945 13865 tablet_service.cc:1514] Processing DeleteTablet for tablet e720a00bd9b5404aa7bffd21d7000256 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:02:21 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:59924
I20260808 23:02:21.727967 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 13563
I20260808 23:02:21.730438 14233 tablet_replica.cc:330] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13: stopping tablet replica
I20260808 23:02:21.730561 14233 raft_consensus.cc:2229] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:02:21.730695 14233 raft_consensus.cc:2258] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:02:21.732192 14235 tablet_replica.cc:330] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d: stopping tablet replica
I20260808 23:02:21.732342 14235 raft_consensus.cc:2229] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:02:21.732486 14235 raft_consensus.cc:2258] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:02:21.734225 14233 ts_tablet_manager.cc:1905] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:02:21.737282 14233 ts_tablet_manager.cc:1918] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:02:21.737404 14233 log.cc:1196] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/e720a00bd9b5404aa7bffd21d7000256
I20260808 23:02:21.737780 14233 ts_tablet_manager.cc:1939] T e720a00bd9b5404aa7bffd21d7000256 P 95875a4556fa43c5b39f640233a9fe13: Deleting consensus metadata
I20260808 23:02:21.738335 13438 catalog_manager.cc:4841] TS 95875a4556fa43c5b39f640233a9fe13 (127.23.194.3:43727): tablet e720a00bd9b5404aa7bffd21d7000256 (table test-table [id=dcf2fd4c734b4f418625ee905c73c66f]) successfully deleted
I20260808 23:02:21.739198 14235 ts_tablet_manager.cc:1905] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:02:21.740799 14235 ts_tablet_manager.cc:1918] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:02:21.740876 14235 log.cc:1196] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/e720a00bd9b5404aa7bffd21d7000256
I20260808 23:02:21.741130 14235 ts_tablet_manager.cc:1939] T e720a00bd9b5404aa7bffd21d7000256 P 686c6bae2d95420bb3911980f6dde11d: Deleting consensus metadata
I20260808 23:02:21.743182 13434 catalog_manager.cc:4841] TS 686c6bae2d95420bb3911980f6dde11d (127.23.194.2:43531): tablet e720a00bd9b5404aa7bffd21d7000256 (table test-table [id=dcf2fd4c734b4f418625ee905c73c66f]) successfully deleted
W20260808 23:02:21.748011 13438 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.194.1:34835 (error 108)
I20260808 23:02:21.748517 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 13769
I20260808 23:02:21.748574 14197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:21.748412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:34835 (local address 127.0.0.1:35636)
0808 23:02:21.748481 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:21.748536 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.1:34835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 23:02:21.753921 13438 catalog_manager.cc:4584] TS 1228fa7b2dab4969ad74438a5495e98f (127.23.194.1:34835): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e720a00bd9b5404aa7bffd21d7000256: Network error: Client connection negotiation failed: client connection to 127.23.194.1:34835: connect: Connection refused (error 111)
I20260808 23:02:21.759115 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 13979
I20260808 23:02:21.771546 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 13410
I20260808 23:02:21.782821 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:34697
--webserver_interface=127.23.194.62
--webserver_port=32807
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:34697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:02:21.863410 14237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:21.863610 14237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:21.863629 14237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:21.864691 14237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:02:21.864723 14237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:21.864735 14237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:21.864746 14237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:02:21.864766 14237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:02:21.866235 14237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:34697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:34697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=32807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:21.866426 14237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:21.866600 14237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:21.875234 14249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:21.875260 14257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:21.875293 14246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:21.875510 14237 server_base.cc:1034] running on GCE node
I20260808 23:02:21.875762 14237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:21.876055 14237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:21.877197 14237 hybrid_clock.cc:648] HybridClock initialized: now 1786230141877184 us; error 28 us; skew 500 ppm
Aug 08 23:02:21 dist-test-slave-f0tn krb5kdc[13389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230141, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:21.883961 14237 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:02:21.884379 14237 webserver.cc:460] Webserver started at http://127.23.194.62:32807/ using document root <none> and password file <none>
I20260808 23:02:21.884590 14237 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:21.884658 14237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:21.886018 14237 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 23:02:21.886606 14266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:21.886755 14237 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:02:21.886842 14237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "4041bcd342cc4f78a338ed497066a169"
format_stamp: "Formatted at 2026-08-08 23:02:21 on dist-test-slave-f0tn"
server_key: "64a98a99920b48b623070e54e1c7fb6c"
server_key_iv: "703acac7db7aa1361d95b12fb6431746"
server_key_version: "encryptionkey@0"
I20260808 23:02:21.887214 14237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:21.905100 14237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:21.906970 14237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:21.907109 14237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:21.911027 14237 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:34697
I20260808 23:02:21.911063 14348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:34697 every 8 connection(s)
I20260808 23:02:21.911423 14237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:02:21.913017 14349 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:02:21.913735 14349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169: Bootstrap starting.
I20260808 23:02:21.918011 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 14237
I20260808 23:02:21.918356 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:34835
--local_ip_for_outbound_sockets=127.23.194.1
--tserver_master_addrs=127.23.194.62:34697
--webserver_port=46781
--webserver_interface=127.23.194.1
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:02:21.923156 14349 log.cc:824] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:21.925051 14349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:02:21.925324 14349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169: Bootstrap complete.
I20260808 23:02:21.927536 14349 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4041bcd342cc4f78a338ed497066a169" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34697 } }
I20260808 23:02:21.927757 14349 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4041bcd342cc4f78a338ed497066a169, State: Initialized, Role: FOLLOWER
I20260808 23:02:21.927896 14349 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4041bcd342cc4f78a338ed497066a169" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34697 } }
I20260808 23:02:21.927960 14349 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:02:21.927997 14349 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:02:21.928046 14349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 1 FOLLOWER]: Advancing to term 2
I20260808 23:02:21.929035 14349 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4041bcd342cc4f78a338ed497066a169" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34697 } }
I20260808 23:02:21.929122 14349 leader_election.cc:302] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4041bcd342cc4f78a338ed497066a169; no voters:
I20260808 23:02:21.929273 14349 leader_election.cc:288] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 23:02:21.929330 14355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 23:02:21.929467 14355 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 2 LEADER]: Becoming Leader. State: Replica: 4041bcd342cc4f78a338ed497066a169, State: Running, Role: LEADER
I20260808 23:02:21.929582 14349 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:02:21.929632 14355 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4041bcd342cc4f78a338ed497066a169" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34697 } }
I20260808 23:02:21.929934 14358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4041bcd342cc4f78a338ed497066a169" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4041bcd342cc4f78a338ed497066a169" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34697 } } }
I20260808 23:02:21.930017 14358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [sys.catalog]: This master's current role is: LEADER
I20260808 23:02:21.929962 14355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4041bcd342cc4f78a338ed497066a169. Latest consensus state: current_term: 2 leader_uuid: "4041bcd342cc4f78a338ed497066a169" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4041bcd342cc4f78a338ed497066a169" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34697 } } }
I20260808 23:02:21.930078 14355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [sys.catalog]: This master's current role is: LEADER
I20260808 23:02:21.930414 14366 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:02:21.931233 14366 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:02:21.931349 14366 catalog_manager.cc:1259] Loaded cluster ID: fa40ffa10f944ec68032d3520d6b0030
I20260808 23:02:21.931569 14366 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:02:21.931792 14366 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 23:02:21.934510 14366 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 23:02:21.938423 14366 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:02:21.938612 14366 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169: Loaded TSK: 0
I20260808 23:02:21.938818 14366 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 23:02:22.008668 14353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:22.008860 14353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:22.008890 14353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:22.010067 14353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:22.010113 14353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:22.010146 14353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:02:22.011557 14353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:34835
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=46781
--tserver_master_addrs=127.23.194.62:34697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:22.011791 14353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:22.011999 14353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:22.017920 14396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:22.017920 14391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:22.017897 14390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:22.018114 14353 server_base.cc:1034] running on GCE node
I20260808 23:02:22.018392 14353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:22.018666 14353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:22.019856 14353 hybrid_clock.cc:648] HybridClock initialized: now 1786230142019837 us; error 31 us; skew 500 ppm
Aug 08 23:02:22 dist-test-slave-f0tn krb5kdc[13389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230142, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:22.022943 14353 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 23:02:22.023459 14353 webserver.cc:460] Webserver started at http://127.23.194.1:46781/ using document root <none> and password file <none>
I20260808 23:02:22.023729 14353 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:22.023805 14353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:22.025449 14353 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:02:22.026194 14407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:22.026360 14353 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:02:22.026435 14353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "1228fa7b2dab4969ad74438a5495e98f"
format_stamp: "Formatted at 2026-08-08 23:02:21 on dist-test-slave-f0tn"
server_key: "7d2867380c386527f52f56945fb81d5c"
server_key_iv: "c0b226ee360cecf9a6827bc4189b71e0"
server_key_version: "encryptionkey@0"
I20260808 23:02:22.026757 14353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:22.053880 14353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:22.056088 14353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:22.056222 14353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:22.056449 14353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:02:22.056907 14420 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 23:02:22.058760 14353 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 23:02:22.058804 14353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 23:02:22.058836 14353 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 23:02:22.059429 14353 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 23:02:22.059463 14353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 23:02:22.059542 14420 tablet_bootstrap.cc:492] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f: Bootstrap starting.
I20260808 23:02:22.065323 14353 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:34835
I20260808 23:02:22.065411 14587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:34835 every 8 connection(s)
I20260808 23:02:22.065716 14353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:02:22.069854 14420 log.cc:824] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:22.072207 14420 tablet_bootstrap.cc:492] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:02:22.072567 14420 tablet_bootstrap.cc:492] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f: Bootstrap complete.
Aug 08 23:02:22 dist-test-slave-f0tn krb5kdc[13389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230142, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:22.073335 14420 ts_tablet_manager.cc:1397] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260808 23:02:22.074420 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 14353
I20260808 23:02:22.074455 14420 raft_consensus.cc:348] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } }
I20260808 23:02:22.074736 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:43531
--local_ip_for_outbound_sockets=127.23.194.2
--tserver_master_addrs=127.23.194.62:34697
--webserver_port=43405
--webserver_interface=127.23.194.2
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:02:22.074744 14420 raft_consensus.cc:729] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1228fa7b2dab4969ad74438a5495e98f, State: Initialized, Role: FOLLOWER
I20260808 23:02:22.074915 14420 consensus_queue.cc:260] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } }
I20260808 23:02:22.075201 14420 ts_tablet_manager.cc:1428] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 23:02:22.076674 14592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.066540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.1:60713)
0808 23:02:22.066741 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:22.066788 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:22.067272 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:22.067687 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:22.067691 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:22.067962 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:22.068153 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:22.068158 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.069239 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.069241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:22.070104 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:22.070107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.070675 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.070676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:22.070801 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:22.071334 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:22.071342 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:22.072946 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:22.074988 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:22.074995 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:22.074997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:22.075254 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:22.075526 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:22.075527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:22.075528 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:22.075666 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:22.076062 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:22.076069 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:22.076220 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 23:02:22.076326 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 23:02:22.076498 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":113,"threads_started":1}
I20260808 23:02:22.076742 14593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.066649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:60713 (local address 127.23.194.62:34697)
0808 23:02:22.066844 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:22.066852 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:22.066891 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:02:22.067479 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:22.067484 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:22.067523 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:22.067644 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:22.068270 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.069124 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.070183 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.070572 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.070670 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:22.073047 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:22.073051 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:22.073051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:22.073062 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:22.074898 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:22.075343 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:22.075346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:22.075347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:22.075457 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:22.075738 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:22.075739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:22.075739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:22.075972 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:22.076061 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:22.076378 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 23:02:22.076540 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":98,"threads_started":1}
I20260808 23:02:22.077410 14588 heartbeater.cc:347] Connected to a master server at 127.23.194.62:34697
I20260808 23:02:22.077512 14588 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:22.077678 14588 heartbeater.cc:511] Master 127.23.194.62:34697 requested a full tablet report, sending...
I20260808 23:02:22.078430 14292 ts_manager.cc:194] Registered new tserver with Master: 1228fa7b2dab4969ad74438a5495e98f (127.23.194.1:34835)
I20260808 23:02:22.079593 14292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:60713
I20260808 23:02:22.080428 14588 heartbeater.cc:503] Master 127.23.194.62:34697 was elected leader, sending a full tablet report...
I20260808 23:02:22.084933 14601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.079090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35650 (local address 127.23.194.1:34835)
0808 23:02:22.079246 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:22.079250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:22.081607 (+ 2357us) server_negotiation.cc:407] Connection header received
0808 23:02:22.081708 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:22.081710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:22.081722 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:22.081798 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:02:22.081967 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.082587 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.084097 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.084543 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.084628 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:22.084683 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:22.084858 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 23:02:22.084887 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260808 23:02:22.084991 14600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.079031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:34835 (local address 127.0.0.1:35650)
0808 23:02:22.081493 (+ 2462us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:22.081535 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:22.081620 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:22.081823 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:22.081827 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:22.081829 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:02:22.081899 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:22.081901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.082666 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.082667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:22.083907 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:22.083911 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.084648 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.084649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:22.084748 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:22.084763 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 23:02:22.084851 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 23:02:22.084912 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2429,"thread_start_us":88,"threads_started":1}
I20260808 23:02:22.085445 14482 tablet_service.cc:1514] Processing DeleteTablet for tablet e720a00bd9b5404aa7bffd21d7000256 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:02:21 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:35650
I20260808 23:02:22.085883 14602 tablet_replica.cc:330] stopping tablet replica
I20260808 23:02:22.085995 14602 raft_consensus.cc:2229] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:02:22.086073 14602 raft_consensus.cc:2258] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:02:22.086403 14602 ts_tablet_manager.cc:1905] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:02:22.088891 14602 ts_tablet_manager.cc:1918] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:02:22.088974 14602 log.cc:1196] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/wal/wals/e720a00bd9b5404aa7bffd21d7000256
I20260808 23:02:22.089227 14602 ts_tablet_manager.cc:1939] T e720a00bd9b5404aa7bffd21d7000256 P 1228fa7b2dab4969ad74438a5495e98f: Deleting consensus metadata
I20260808 23:02:22.089761 14270 catalog_manager.cc:4841] TS 1228fa7b2dab4969ad74438a5495e98f (127.23.194.1:34835): tablet e720a00bd9b5404aa7bffd21d7000256 (table test-table [id=dcf2fd4c734b4f418625ee905c73c66f]) successfully deleted
W20260808 23:02:22.162906 14599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:22.163130 14599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:22.163156 14599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:22.164583 14599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:22.164639 14599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:22.164669 14599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:02:22.166405 14599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:43531
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=43405
--tserver_master_addrs=127.23.194.62:34697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:22.166631 14599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:22.166857 14599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:22.171690 14611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:22.171686 14612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:22.171723 14616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:22.171758 14599 server_base.cc:1034] running on GCE node
I20260808 23:02:22.172086 14599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:22.172309 14599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:22.173427 14599 hybrid_clock.cc:648] HybridClock initialized: now 1786230142173410 us; error 32 us; skew 500 ppm
Aug 08 23:02:22 dist-test-slave-f0tn krb5kdc[13389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230142, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:22.175714 14599 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 23:02:22.176110 14599 webserver.cc:460] Webserver started at http://127.23.194.2:43405/ using document root <none> and password file <none>
I20260808 23:02:22.176306 14599 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:22.176366 14599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:22.177675 14599 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:02:22.178182 14628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:22.178313 14599 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:22.178393 14599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "686c6bae2d95420bb3911980f6dde11d"
format_stamp: "Formatted at 2026-08-08 23:02:21 on dist-test-slave-f0tn"
server_key: "ee7aae15ef2034ac70b918f02ebd95a3"
server_key_iv: "49ecd2815841439a70b89362cfc2f75d"
server_key_version: "encryptionkey@0"
I20260808 23:02:22.178694 14599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:22.189731 14599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:22.191370 14599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:22.191493 14599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:22.191708 14599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:02:22.191982 14599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:02:22.192015 14599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:22.192040 14599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:02:22.192058 14599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:22.197948 14599 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:43531
I20260808 23:02:22.197981 14794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:43531 every 8 connection(s)
I20260808 23:02:22.198303 14599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:02:22.201297 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 14599
I20260808 23:02:22.201666 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:43727
--local_ip_for_outbound_sockets=127.23.194.3
--tserver_master_addrs=127.23.194.62:34697
--webserver_port=44753
--webserver_interface=127.23.194.3
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:02:22 dist-test-slave-f0tn krb5kdc[13389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230142, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:22.207491 14593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.199024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:32921 (local address 127.23.194.62:34697)
0808 23:02:22.199088 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:22.199090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:22.199265 (+ 175us) server_negotiation.cc:407] Connection header received
0808 23:02:22.199716 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:22.199717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:22.199726 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:22.199778 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:22.200660 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.201167 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.202518 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.202725 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.202780 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:22.205040 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:22.205041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:22.205041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:22.205048 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:22.206290 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:22.206649 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:22.206650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:22.206650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:22.206700 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:22.206903 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:22.206904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:22.206904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:22.207007 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:22.207062 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:22.207310 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 23:02:22.207402 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 23:02:22.207629 14797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.198993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.2:32921)
0808 23:02:22.199146 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:22.199185 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:22.199639 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:22.199830 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:22.199833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:22.200384 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:22.200570 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:22.200574 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.201253 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.201254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:22.202424 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:22.202427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.202807 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.202808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:22.202944 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:22.203521 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:22.203529 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:22.204951 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:22.206363 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:22.206370 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:22.206372 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:22.206576 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:22.206753 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:22.206754 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:22.206754 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:22.206848 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:22.207058 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:22.207065 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:22.207196 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 23:02:22.207280 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 23:02:22.207422 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260808 23:02:22.208079 14795 heartbeater.cc:347] Connected to a master server at 127.23.194.62:34697
I20260808 23:02:22.208191 14795 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:22.208395 14795 heartbeater.cc:511] Master 127.23.194.62:34697 requested a full tablet report, sending...
I20260808 23:02:22.208815 14292 ts_manager.cc:194] Registered new tserver with Master: 686c6bae2d95420bb3911980f6dde11d (127.23.194.2:43531)
I20260808 23:02:22.209167 14292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:32921
W20260808 23:02:22.286470 14798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:22.286681 14798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:22.286706 14798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:22.287868 14798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:22.287914 14798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:22.287941 14798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:02:22.289427 14798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:43727
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=44753
--tserver_master_addrs=127.23.194.62:34697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:22.289680 14798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:22.289901 14798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:22.294785 14810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:22.294798 14807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:22.294790 14804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:22.294986 14798 server_base.cc:1034] running on GCE node
I20260808 23:02:22.295245 14798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:22.295439 14798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:22.296571 14798 hybrid_clock.cc:648] HybridClock initialized: now 1786230142296561 us; error 31 us; skew 500 ppm
Aug 08 23:02:22 dist-test-slave-f0tn krb5kdc[13389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230142, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:22.299005 14798 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 23:02:22.299394 14798 webserver.cc:460] Webserver started at http://127.23.194.3:44753/ using document root <none> and password file <none>
I20260808 23:02:22.299563 14798 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:22.299614 14798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:22.300846 14798 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:02:22.301362 14820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:22.301560 14798 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:22.301646 14798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "95875a4556fa43c5b39f640233a9fe13"
format_stamp: "Formatted at 2026-08-08 23:02:21 on dist-test-slave-f0tn"
server_key: "d88f02659256f77b77348ffabc26534a"
server_key_iv: "a80481bd6259776befd898c9fd4a0e63"
server_key_version: "encryptionkey@0"
I20260808 23:02:22.301972 14798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:22.326093 14798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:22.328202 14798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:22.328323 14798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:22.328529 14798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:02:22.328815 14798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:02:22.328846 14798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:22.328867 14798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:02:22.328886 14798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:22.334770 14798 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:43727
I20260808 23:02:22.334815 14998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:43727 every 8 connection(s)
I20260808 23:02:22.335155 14798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 23:02:22.337704 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 14798
Aug 08 23:02:22 dist-test-slave-f0tn krb5kdc[13389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230142, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:22.344777 15003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.338100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58994 (local address 127.23.194.62:34697)
0808 23:02:22.338208 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:22.338209 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:22.338275 (+ 66us) server_negotiation.cc:407] Connection header received
0808 23:02:22.338337 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:22.338338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:22.338347 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:22.338406 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:22.339038 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.339827 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.340772 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.341021 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.341090 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:22.342337 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:22.342338 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:22.342339 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:22.342345 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:22.343757 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:22.344062 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:22.344063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:22.344064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:22.344118 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:22.344296 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:22.344296 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:22.344297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:22.344397 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:22.344547 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:22.344582 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 23:02:22.344681 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":62,"threads_started":1}
I20260808 23:02:22.345489 14593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.336031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:56661 (local address 127.23.194.62:34697)
0808 23:02:22.336132 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:22.336134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:22.336310 (+ 176us) server_negotiation.cc:407] Connection header received
0808 23:02:22.336815 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:22.336816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:22.336827 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:22.336884 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:22.337969 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.338698 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.340091 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.340384 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.340456 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:22.342869 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:22.342871 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:22.342871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:22.342877 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:22.344194 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:22.344558 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:22.344559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:22.344559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:22.344607 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:22.344852 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:22.344852 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:22.344853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:22.344953 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:22.345007 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:22.345319 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 23:02:22.345407 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 23:02:22.345709 15001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.335961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.3:56661)
0808 23:02:22.336178 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:22.336223 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:22.336729 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:22.336951 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:22.336955 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:22.337618 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:22.337866 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:22.337873 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.338807 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.338808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:22.340007 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:22.340010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.340492 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.340493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:22.340606 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:22.341119 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:22.341129 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:22.342780 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:22.344270 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:22.344278 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:22.344280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:22.344492 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:22.344667 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:22.344668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:22.344668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:22.344802 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:22.345010 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:22.345017 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:22.345178 (+ 161us) client_negotiation.cc:769] Sending connection context
0808 23:02:22.345286 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 23:02:22.345480 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":105,"threads_started":1}
I20260808 23:02:22.346133 14999 heartbeater.cc:347] Connected to a master server at 127.23.194.62:34697
I20260808 23:02:22.346226 14999 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:22.346401 14999 heartbeater.cc:511] Master 127.23.194.62:34697 requested a full tablet report, sending...
I20260808 23:02:22.346745 14292 ts_manager.cc:194] Registered new tserver with Master: 95875a4556fa43c5b39f640233a9fe13 (127.23.194.3:43727)
I20260808 23:02:22.347078 14292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:56661
I20260808 23:02:22.356233 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:02:22.358968 14593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.356552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58996 (local address 127.23.194.62:34697)
0808 23:02:22.356597 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:22.356598 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:22.356744 (+ 146us) server_negotiation.cc:407] Connection header received
0808 23:02:22.356814 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:22.356815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:22.356855 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:22.356912 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:02:22.357024 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.357552 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.358283 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.358477 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.358529 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:22.358687 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:02:22.358782 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:02:22.358847 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:22.358885 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 23:02:22.358902 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 23:02:22.359261 14292 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:58996:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:02:22.360812 14292 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:02:22.362910 14482 tablet_service.cc:1467] Processing CreateTablet for tablet d42573bd05f24bfbbaae40595e619be0 (DEFAULT_TABLE table=test-table [id=60a2a3a6a6ef4f63b9ffc4bfa1b4eca8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:02:22.363122 14482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d42573bd05f24bfbbaae40595e619be0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:02:22.364678 14420 tablet_bootstrap.cc:492] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f: Bootstrap starting.
I20260808 23:02:22.364964 14420 tablet_bootstrap.cc:654] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f: Neither blocks nor log segments found. Creating new log.
I20260808 23:02:22.365480 14420 tablet_bootstrap.cc:492] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f: No bootstrap required, opened a new log
I20260808 23:02:22.365532 14420 ts_tablet_manager.cc:1397] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 23:02:22.365645 14420 raft_consensus.cc:348] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } }
I20260808 23:02:22.365726 14420 raft_consensus.cc:374] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:02:22.365775 14420 raft_consensus.cc:729] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1228fa7b2dab4969ad74438a5495e98f, State: Initialized, Role: FOLLOWER
I20260808 23:02:22.365824 14420 consensus_queue.cc:260] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } }
I20260808 23:02:22.365917 14420 ts_tablet_manager.cc:1428] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:22.366740 14600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.362577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:43727 (local address 127.0.0.1:50594)
0808 23:02:22.362627 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:22.362638 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:22.362682 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:22.363223 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:22.363224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:22.363226 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:02:22.363263 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:22.363265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.364463 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.364464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:22.365507 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:22.365509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.366500 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.366501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:22.366561 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:22.366564 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:02:22.366613 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 23:02:22.366664 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 23:02:22.366747 15006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.362621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:43531 (local address 127.0.0.1:59930)
0808 23:02:22.362876 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:22.362886 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:22.362928 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:22.363169 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:22.363170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:22.363172 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:02:22.363213 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:22.363215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.364291 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.364292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:22.365439 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:22.365442 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.366551 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.366552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:22.366602 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:22.366605 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:02:22.366652 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 23:02:22.366683 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":32,"thread_start_us":112,"threads_started":1}
I20260808 23:02:22.366781 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.362855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59930 (local address 127.23.194.2:43531)
0808 23:02:22.363016 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:22.363021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:22.363033 (+ 12us) server_negotiation.cc:407] Connection header received
0808 23:02:22.363072 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:22.363075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:22.363090 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:22.363170 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:02:22.363263 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.364198 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.365595 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.366449 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.366547 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:22.366608 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:22.366667 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 23:02:22.366705 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260808 23:02:22.366967 15007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.362665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50594 (local address 127.23.194.3:43727)
0808 23:02:22.363054 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:22.363058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:22.363067 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:02:22.363102 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:22.363104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:22.363119 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:22.363331 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:02:22.363403 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.364365 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.365588 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.366328 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.366418 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:22.366481 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:22.366848 (+ 367us) server_negotiation.cc:299] Negotiation successful
0808 23:02:22.366892 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":82,"threads_started":1}
I20260808 23:02:22.367458 14694 tablet_service.cc:1467] Processing CreateTablet for tablet d42573bd05f24bfbbaae40595e619be0 (DEFAULT_TABLE table=test-table [id=60a2a3a6a6ef4f63b9ffc4bfa1b4eca8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:02:22.367650 14886 tablet_service.cc:1467] Processing CreateTablet for tablet d42573bd05f24bfbbaae40595e619be0 (DEFAULT_TABLE table=test-table [id=60a2a3a6a6ef4f63b9ffc4bfa1b4eca8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:02:22.367794 14694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d42573bd05f24bfbbaae40595e619be0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:02:22.367972 14886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d42573bd05f24bfbbaae40595e619be0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:02:22.370342 15012 tablet_bootstrap.cc:492] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d: Bootstrap starting.
I20260808 23:02:22.370627 15013 tablet_bootstrap.cc:492] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13: Bootstrap starting.
I20260808 23:02:22.371062 15012 tablet_bootstrap.cc:654] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d: Neither blocks nor log segments found. Creating new log.
I20260808 23:02:22.371232 15013 tablet_bootstrap.cc:654] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13: Neither blocks nor log segments found. Creating new log.
I20260808 23:02:22.371519 15012 log.cc:824] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:22.371681 15013 log.cc:824] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:22.372279 15012 tablet_bootstrap.cc:492] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d: No bootstrap required, opened a new log
I20260808 23:02:22.372341 15013 tablet_bootstrap.cc:492] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13: No bootstrap required, opened a new log
I20260808 23:02:22.372342 15012 ts_tablet_manager.cc:1397] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:02:22.372406 15013 ts_tablet_manager.cc:1397] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:02:22.373726 15012 raft_consensus.cc:348] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } }
I20260808 23:02:22.373754 15013 raft_consensus.cc:348] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } }
I20260808 23:02:22.373862 15013 raft_consensus.cc:374] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:02:22.373850 15012 raft_consensus.cc:374] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:02:22.373880 15013 raft_consensus.cc:729] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95875a4556fa43c5b39f640233a9fe13, State: Initialized, Role: FOLLOWER
I20260808 23:02:22.373883 15012 raft_consensus.cc:729] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 686c6bae2d95420bb3911980f6dde11d, State: Initialized, Role: FOLLOWER
I20260808 23:02:22.373996 15013 consensus_queue.cc:260] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } }
I20260808 23:02:22.374010 15012 consensus_queue.cc:260] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } }
I20260808 23:02:22.374287 15012 ts_tablet_manager.cc:1428] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 23:02:22.374315 14795 heartbeater.cc:503] Master 127.23.194.62:34697 was elected leader, sending a full tablet report...
I20260808 23:02:22.374266 14999 heartbeater.cc:503] Master 127.23.194.62:34697 was elected leader, sending a full tablet report...
I20260808 23:02:22.374251 15013 ts_tablet_manager.cc:1428] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 23:02:22.401517 15019 raft_consensus.cc:482] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:02:22.401641 15019 raft_consensus.cc:504] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } }
I20260808 23:02:22.401983 15019 leader_election.cc:288] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95875a4556fa43c5b39f640233a9fe13 (127.23.194.3:43727), 1228fa7b2dab4969ad74438a5495e98f (127.23.194.1:34835)
I20260808 23:02:22.405694 14601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.402387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:56453 (local address 127.23.194.1:34835)
0808 23:02:22.402452 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:22.402454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:22.402579 (+ 125us) server_negotiation.cc:407] Connection header received
0808 23:02:22.402650 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:22.402652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:22.402661 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:22.402714 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:02:22.402806 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.403486 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.404730 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.405315 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.405406 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:22.405413 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:22.405582 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 23:02:22.405630 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 23:02:22.405694 14797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.402195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:43727 (local address 127.23.194.2:46693)
0808 23:02:22.402266 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:22.402280 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:22.402336 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:22.402461 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:22.402462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:22.402464 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:02:22.402551 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:22.402553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.403443 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.403444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:22.404653 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:22.404656 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.405401 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.405402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:22.405518 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:22.405520 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 23:02:22.405565 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 23:02:22.405601 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 23:02:22.405748 15007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.402298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:46693 (local address 127.23.194.3:43727)
0808 23:02:22.402349 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:22.402351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:22.402359 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:02:22.402388 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:22.402388 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:22.402397 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:22.402448 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:02:22.402619 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.403370 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.404730 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.405301 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.405401 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:22.405410 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:22.405648 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 23:02:22.405691 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 23:02:22.406067 15020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.402324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:34835 (local address 127.23.194.2:56453)
0808 23:02:22.402512 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:22.402522 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:22.402590 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:22.402719 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:22.402719 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:22.402720 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:02:22.402766 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:22.402767 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.403551 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.403552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:22.404661 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:22.404662 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.405401 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.405402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:22.405500 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:22.405505 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:02:22.405560 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 23:02:22.405600 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":5,"thread_start_us":115,"threads_started":1}
I20260808 23:02:22.406090 14926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42573bd05f24bfbbaae40595e619be0" candidate_uuid: "686c6bae2d95420bb3911980f6dde11d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95875a4556fa43c5b39f640233a9fe13" is_pre_election: true
I20260808 23:02:22.406245 14926 raft_consensus.cc:2454] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 686c6bae2d95420bb3911980f6dde11d in term 0.
I20260808 23:02:22.406466 14631 leader_election.cc:302] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 686c6bae2d95420bb3911980f6dde11d, 95875a4556fa43c5b39f640233a9fe13; no voters:
I20260808 23:02:22.406615 15019 raft_consensus.cc:2781] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:02:22.406581 14516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42573bd05f24bfbbaae40595e619be0" candidate_uuid: "686c6bae2d95420bb3911980f6dde11d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1228fa7b2dab4969ad74438a5495e98f" is_pre_election: true
I20260808 23:02:22.406678 15019 raft_consensus.cc:482] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:02:22.406709 15019 raft_consensus.cc:3037] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:02:22.406731 14516 raft_consensus.cc:2454] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 686c6bae2d95420bb3911980f6dde11d in term 0.
I20260808 23:02:22.407418 15019 raft_consensus.cc:504] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } }
I20260808 23:02:22.407523 15019 leader_election.cc:288] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [CANDIDATE]: Term 1 election: Requested vote from peers 95875a4556fa43c5b39f640233a9fe13 (127.23.194.3:43727), 1228fa7b2dab4969ad74438a5495e98f (127.23.194.1:34835)
I20260808 23:02:22.407704 14926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42573bd05f24bfbbaae40595e619be0" candidate_uuid: "686c6bae2d95420bb3911980f6dde11d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95875a4556fa43c5b39f640233a9fe13"
I20260808 23:02:22.407711 14516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42573bd05f24bfbbaae40595e619be0" candidate_uuid: "686c6bae2d95420bb3911980f6dde11d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1228fa7b2dab4969ad74438a5495e98f"
I20260808 23:02:22.407783 14926 raft_consensus.cc:3037] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:02:22.407783 14516 raft_consensus.cc:3037] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:02:22.408622 14926 raft_consensus.cc:2454] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 686c6bae2d95420bb3911980f6dde11d in term 1.
I20260808 23:02:22.408622 14516 raft_consensus.cc:2454] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 686c6bae2d95420bb3911980f6dde11d in term 1.
I20260808 23:02:22.408800 14631 leader_election.cc:302] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 686c6bae2d95420bb3911980f6dde11d, 95875a4556fa43c5b39f640233a9fe13; no voters:
I20260808 23:02:22.408913 15019 raft_consensus.cc:2781] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:02:22.409039 15019 raft_consensus.cc:686] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [term 1 LEADER]: Becoming Leader. State: Replica: 686c6bae2d95420bb3911980f6dde11d, State: Running, Role: LEADER
I20260808 23:02:22.409114 15019 consensus_queue.cc:237] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } }
I20260808 23:02:22.409945 14292 catalog_manager.cc:5495] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d reported cstate change: term changed from 0 to 1, leader changed from <none> to 686c6bae2d95420bb3911980f6dde11d (127.23.194.2). New cstate: current_term: 1 leader_uuid: "686c6bae2d95420bb3911980f6dde11d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260808 23:02:22.424628 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.422677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59942 (local address 127.23.194.2:43531)
0808 23:02:22.422726 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:22.422728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:22.422749 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:02:22.422780 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:22.422780 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:22.422796 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:22.422847 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:02:22.422940 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.423395 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.424049 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:22.424245 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:22.424294 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:22.424380 (+ 86us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:02:22.424472 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:02:22.424521 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:22.424540 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 23:02:22.424556 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 23:02:22.426460 14516 raft_consensus.cc:1261] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Refusing update from remote peer 686c6bae2d95420bb3911980f6dde11d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:02:22.426467 14926 raft_consensus.cc:1261] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13 [term 1 FOLLOWER]: Refusing update from remote peer 686c6bae2d95420bb3911980f6dde11d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:02:22.426788 15019 consensus_queue.cc:1035] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [LEADER]: Connected to new peer: Peer: permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:02:22.426937 15022 consensus_queue.cc:1035] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:02:22.429128 15025 mvcc.cc:205] Tried to move back new op lower bound from 7316398663376175104 to 7316398663308337152. Current Snapshot: MvccSnapshot[applied={T|T < 7316398663376175104}]
I20260808 23:02:22.429633 15026 mvcc.cc:205] Tried to move back new op lower bound from 7316398663376175104 to 7316398663308337152. Current Snapshot: MvccSnapshot[applied={T|T < 7316398663376175104 or (T in {7316398663376175104})}]
I20260808 23:02:22.429965 15028 mvcc.cc:205] Tried to move back new op lower bound from 7316398663376175104 to 7316398663308337152. Current Snapshot: MvccSnapshot[applied={T|T < 7316398663376175104}]
I20260808 23:02:22.431516 14290 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:58996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:02:22.431608 14290 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:58996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:02:22.432332 14290 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169: Sending DeleteTablet for 3 replicas of tablet d42573bd05f24bfbbaae40595e619be0
I20260808 23:02:22.432641 14694 tablet_service.cc:1514] Processing DeleteTablet for tablet d42573bd05f24bfbbaae40595e619be0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:02:22 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:59930
I20260808 23:02:22.432706 14886 tablet_service.cc:1514] Processing DeleteTablet for tablet d42573bd05f24bfbbaae40595e619be0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:02:22 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:50594
I20260808 23:02:22.432890 14482 tablet_service.cc:1514] Processing DeleteTablet for tablet d42573bd05f24bfbbaae40595e619be0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:02:22 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:35650
I20260808 23:02:22.432973 14602 tablet_replica.cc:330] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f: stopping tablet replica
I20260808 23:02:22.432982 15040 tablet_replica.cc:330] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d: stopping tablet replica
I20260808 23:02:22.432986 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 14353
I20260808 23:02:22.433018 14602 raft_consensus.cc:2229] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:02:22.433063 15040 raft_consensus.cc:2229] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:02:22.433070 14602 raft_consensus.cc:2258] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:02:22.433162 15040 raft_consensus.cc:2258] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:02:22.444461 15040 ts_tablet_manager.cc:1905] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:02:22.445526 15042 tablet_replica.cc:330] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13: stopping tablet replica
I20260808 23:02:22.445716 15042 raft_consensus.cc:2229] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:02:22.445917 15042 raft_consensus.cc:2258] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:02:22.446249 15040 ts_tablet_manager.cc:1918] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:02:22.446369 15040 log.cc:1196] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/d42573bd05f24bfbbaae40595e619be0
I20260808 23:02:22.446486 15042 ts_tablet_manager.cc:1905] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:02:22.446952 15040 ts_tablet_manager.cc:1939] T d42573bd05f24bfbbaae40595e619be0 P 686c6bae2d95420bb3911980f6dde11d: Deleting consensus metadata
I20260808 23:02:22.447575 14267 catalog_manager.cc:4841] TS 686c6bae2d95420bb3911980f6dde11d (127.23.194.2:43531): tablet d42573bd05f24bfbbaae40595e619be0 (table test-table [id=60a2a3a6a6ef4f63b9ffc4bfa1b4eca8]) successfully deleted
I20260808 23:02:22.448195 15042 ts_tablet_manager.cc:1918] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:02:22.448272 15042 log.cc:1196] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/d42573bd05f24bfbbaae40595e619be0
I20260808 23:02:22.448561 15042 ts_tablet_manager.cc:1939] T d42573bd05f24bfbbaae40595e619be0 P 95875a4556fa43c5b39f640233a9fe13: Deleting consensus metadata
I20260808 23:02:22.449023 14270 catalog_manager.cc:4841] TS 95875a4556fa43c5b39f640233a9fe13 (127.23.194.3:43727): tablet d42573bd05f24bfbbaae40595e619be0 (table test-table [id=60a2a3a6a6ef4f63b9ffc4bfa1b4eca8]) successfully deleted
W20260808 23:02:22.452622 14270 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.194.1:34835 (error 108)
I20260808 23:02:22.452878 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 14599
I20260808 23:02:22.454905 15006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.454761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:34835 (local address 127.0.0.1:35666)
0808 23:02:22.454815 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:22.454869 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.1:34835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 23:02:22.455057 14270 catalog_manager.cc:4584] TS 1228fa7b2dab4969ad74438a5495e98f (127.23.194.1:34835): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d42573bd05f24bfbbaae40595e619be0: Network error: Client connection negotiation failed: client connection to 127.23.194.1:34835: connect: Connection refused (error 111)
I20260808 23:02:22.461140 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 14798
I20260808 23:02:22.471066 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 14237
I20260808 23:02:22.480393 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:34697
--webserver_interface=127.23.194.62
--webserver_port=32807
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:34697
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:02:22.564762 15044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:22.564968 15044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:22.564999 15044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:22.566169 15044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:02:22.566210 15044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:22.566228 15044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:22.566244 15044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:02:22.566260 15044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:02:22.567718 15044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:34697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:34697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=32807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:22.567950 15044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:22.568146 15044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:22.572851 15055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:22.572861 15053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:22.572970 15044 server_base.cc:1034] running on GCE node
W20260808 23:02:22.572861 15052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:22.573279 15044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:22.573583 15044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:22.574719 15044 hybrid_clock.cc:648] HybridClock initialized: now 1786230142574704 us; error 28 us; skew 500 ppm
Aug 08 23:02:22 dist-test-slave-f0tn krb5kdc[13389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230142, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:22.577128 15044 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:02:22.577590 15044 webserver.cc:460] Webserver started at http://127.23.194.62:32807/ using document root <none> and password file <none>
I20260808 23:02:22.577792 15044 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:22.577841 15044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:22.579099 15044 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:02:22.579687 15065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:22.579839 15044 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:02:22.579922 15044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "4041bcd342cc4f78a338ed497066a169"
format_stamp: "Formatted at 2026-08-08 23:02:21 on dist-test-slave-f0tn"
server_key: "64a98a99920b48b623070e54e1c7fb6c"
server_key_iv: "703acac7db7aa1361d95b12fb6431746"
server_key_version: "encryptionkey@0"
I20260808 23:02:22.580241 15044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:22.595896 15044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:22.597900 15044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:22.598052 15044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:22.602330 15044 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:34697
I20260808 23:02:22.602361 15143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:34697 every 8 connection(s)
I20260808 23:02:22.602747 15044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:02:22.604308 15144 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:02:22.604965 15144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169: Bootstrap starting.
I20260808 23:02:22.605444 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 15044
I20260808 23:02:22.605774 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:34835
--local_ip_for_outbound_sockets=127.23.194.1
--tserver_master_addrs=127.23.194.62:34697
--webserver_port=46781
--webserver_interface=127.23.194.1
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:02:22.614454 15144 log.cc:824] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:22.616595 15144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:02:22.616854 15144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169: Bootstrap complete.
I20260808 23:02:22.618950 15144 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4041bcd342cc4f78a338ed497066a169" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34697 } }
I20260808 23:02:22.619168 15144 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4041bcd342cc4f78a338ed497066a169, State: Initialized, Role: FOLLOWER
I20260808 23:02:22.619323 15144 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4041bcd342cc4f78a338ed497066a169" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34697 } }
I20260808 23:02:22.619381 15144 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:02:22.619419 15144 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:02:22.619463 15144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 2 FOLLOWER]: Advancing to term 3
I20260808 23:02:22.620461 15144 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4041bcd342cc4f78a338ed497066a169" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34697 } }
I20260808 23:02:22.620551 15144 leader_election.cc:302] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4041bcd342cc4f78a338ed497066a169; no voters:
I20260808 23:02:22.620704 15144 leader_election.cc:288] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 23:02:22.620748 15148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 23:02:22.620863 15148 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [term 3 LEADER]: Becoming Leader. State: Replica: 4041bcd342cc4f78a338ed497066a169, State: Running, Role: LEADER
I20260808 23:02:22.620963 15144 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:02:22.620954 15148 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4041bcd342cc4f78a338ed497066a169" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34697 } }
I20260808 23:02:22.621233 15150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4041bcd342cc4f78a338ed497066a169" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4041bcd342cc4f78a338ed497066a169" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34697 } } }
I20260808 23:02:22.621272 15148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4041bcd342cc4f78a338ed497066a169. Latest consensus state: current_term: 3 leader_uuid: "4041bcd342cc4f78a338ed497066a169" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4041bcd342cc4f78a338ed497066a169" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34697 } } }
I20260808 23:02:22.621433 15150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [sys.catalog]: This master's current role is: LEADER
I20260808 23:02:22.621450 15148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4041bcd342cc4f78a338ed497066a169 [sys.catalog]: This master's current role is: LEADER
I20260808 23:02:22.621807 15160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:02:22.622608 15160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:02:22.622723 15160 catalog_manager.cc:1259] Loaded cluster ID: fa40ffa10f944ec68032d3520d6b0030
I20260808 23:02:22.622759 15160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 23:02:22.622967 15160 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f08a79f19ad google::LogMessage::Fail() at ??:0
@ 0x7f08a79f488a google::LogMessage::SendToLog() at ??:0
@ 0x7f08a79f14de google::LogMessage::Flush() at ??:0
@ 0x7f08a79f2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f08af9120f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f08af91b8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f08a98bf0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f08a98b4954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f08aa7186db start_thread at ??:0
@ 0x7f08a64ef61f clone at ??:0
W20260808 23:02:22.712046 15146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:22.712314 15146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:22.712350 15146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:22.714001 15146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:22.714068 15146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:22.714108 15146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:02:22.716197 15146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:34835
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=46781
--tserver_master_addrs=127.23.194.62:34697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:22.716450 15146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:22.716728 15146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:22.721957 15192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:22.721974 15191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:22.722218 15195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:22.722215 15146 server_base.cc:1034] running on GCE node
I20260808 23:02:22.722496 15146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:22.722718 15146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:22.723857 15146 hybrid_clock.cc:648] HybridClock initialized: now 1786230142723844 us; error 29 us; skew 500 ppm
Aug 08 23:02:22 dist-test-slave-f0tn krb5kdc[13389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230142, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:22.726382 15146 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 23:02:22.726897 15146 webserver.cc:460] Webserver started at http://127.23.194.1:46781/ using document root <none> and password file <none>
I20260808 23:02:22.727185 15146 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:22.727250 15146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:22.728896 15146 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 23:02:22.729673 15210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:22.729839 15146 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:02:22.729940 15146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "1228fa7b2dab4969ad74438a5495e98f"
format_stamp: "Formatted at 2026-08-08 23:02:21 on dist-test-slave-f0tn"
server_key: "7d2867380c386527f52f56945fb81d5c"
server_key_iv: "c0b226ee360cecf9a6827bc4189b71e0"
server_key_version: "encryptionkey@0"
I20260808 23:02:22.730373 15146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:22.744812 15146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:22.747081 15146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:22.747225 15146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:22.747464 15146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:02:22.747947 15224 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 23:02:22.749362 15146 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 23:02:22.749449 15146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 23:02:22.749482 15146 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 23:02:22.750371 15146 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 23:02:22.750417 15146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 23:02:22.750473 15224 tablet_bootstrap.cc:492] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f: Bootstrap starting.
I20260808 23:02:22.757368 15146 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:34835
I20260808 23:02:22.757630 15370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:34835 every 8 connection(s)
I20260808 23:02:22.757841 15146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:02:22.760284 15224 log.cc:824] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:22.761770 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 15146
I20260808 23:02:22.762094 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:43531
--local_ip_for_outbound_sockets=127.23.194.2
--tserver_master_addrs=127.23.194.62:34697
--webserver_port=43405
--webserver_interface=127.23.194.2
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:02:22.762660 15224 tablet_bootstrap.cc:492] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:02:22.762949 15224 tablet_bootstrap.cc:492] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f: Bootstrap complete.
I20260808 23:02:22.763947 15224 ts_tablet_manager.cc:1397] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.010s
I20260808 23:02:22.764941 15224 raft_consensus.cc:348] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } }
I20260808 23:02:22.765174 15224 raft_consensus.cc:729] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1228fa7b2dab4969ad74438a5495e98f, State: Initialized, Role: FOLLOWER
I20260808 23:02:22.765313 15224 consensus_queue.cc:260] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } }
I20260808 23:02:22.765576 15224 ts_tablet_manager.cc:1428] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f: Time spent starting tablet: real 0.002s user 0.002s sys 0.002s
W20260808 23:02:22.845485 15376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:22.845712 15376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:22.845747 15376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:22.846930 15376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:22.846971 15376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:22.847005 15376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:02:22.848663 15376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:43531
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=43405
--tserver_master_addrs=127.23.194.62:34697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:22.848910 15376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:22.849149 15376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:22.854573 15373 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:02:22.758614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.1:60183)
0808 23:02:22.758801 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:22.758873 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:22.759562 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:22.854248 (+ 94686us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":78,"threads_started":1}
W20260808 23:02:22.854897 15371 heartbeater.cc:644] Failed to heartbeat to 127.23.194.62:34697 (0 consecutive failures): Network error: Failed to ping master at 127.23.194.62:34697: Client connection negotiation failed: client connection to 127.23.194.62:34697: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 23:02:22.855017 15387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:22.855027 15391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:22.855227 15376 server_base.cc:1034] running on GCE node
I20260808 23:02:22.855306 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.855194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.1:50785)
0808 23:02:22.855234 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:22.855276 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260808 23:02:22.855360 15385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:22.855598 15376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:22.855674 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.855592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.1:45317)
0808 23:02:22.855625 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:22.855652 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260808 23:02:22.855808 15371 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 23:02:22.855856 15376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:22.857020 15376 hybrid_clock.cc:648] HybridClock initialized: now 1786230142857005 us; error 29 us; skew 500 ppm
Aug 08 23:02:22 dist-test-slave-f0tn krb5kdc[13389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230142, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:22.859396 15376 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 23:02:22.859797 15376 webserver.cc:460] Webserver started at http://127.23.194.2:43405/ using document root <none> and password file <none>
I20260808 23:02:22.859994 15376 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:22.860035 15376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:22.861239 15376 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 23:02:22.861819 15404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:22.861975 15376 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:22.862051 15376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "686c6bae2d95420bb3911980f6dde11d"
format_stamp: "Formatted at 2026-08-08 23:02:21 on dist-test-slave-f0tn"
server_key: "ee7aae15ef2034ac70b918f02ebd95a3"
server_key_iv: "49ecd2815841439a70b89362cfc2f75d"
server_key_version: "encryptionkey@0"
I20260808 23:02:22.862370 15376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:22.870105 15376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:22.871778 15376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:22.871883 15376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:22.872061 15376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:02:22.872305 15376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:02:22.872334 15376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:22.872359 15376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:02:22.872375 15376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:22.878193 15376 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:43531
I20260808 23:02:22.878263 15564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:43531 every 8 connection(s)
I20260808 23:02:22.878552 15376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:02:22.879524 15570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.879160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.2:47393)
0808 23:02:22.879311 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:22.879388 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
W20260808 23:02:22.879892 15565 heartbeater.cc:644] Failed to heartbeat to 127.23.194.62:34697 (0 consecutive failures): Network error: Failed to ping master at 127.23.194.62:34697: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
I20260808 23:02:22.880184 15570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.880048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.2:34861)
0808 23:02:22.880120 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:22.880155 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 23:02:22.880474 15570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:22.880373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.2:45861)
0808 23:02:22.880426 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:22.880454 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 23:02:22.880618 15565 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 23:02:22.888787 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 15376
I20260808 23:02:22.889030 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:43727
--local_ip_for_outbound_sockets=127.23.194.3
--tserver_master_addrs=127.23.194.62:34697
--webserver_port=44753
--webserver_interface=127.23.194.3
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:02:22.970459 15571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:22.970650 15571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:22.970671 15571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:22.971776 15571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:22.971818 15571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:22.971848 15571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:02:22.973273 15571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:43727
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=44753
--tserver_master_addrs=127.23.194.62:34697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:22.973513 15571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:22.973722 15571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:22.978524 15577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:22.978535 15580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:22.978624 15571 server_base.cc:1034] running on GCE node
W20260808 23:02:22.978535 15578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:22.979009 15571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:22.979235 15571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:22.980366 15571 hybrid_clock.cc:648] HybridClock initialized: now 1786230142980354 us; error 28 us; skew 500 ppm
Aug 08 23:02:22 dist-test-slave-f0tn krb5kdc[13389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230142, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:22.982790 15571 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 23:02:22.983227 15571 webserver.cc:460] Webserver started at http://127.23.194.3:44753/ using document root <none> and password file <none>
I20260808 23:02:22.983443 15571 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:22.983492 15571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:22.984704 15571 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:02:22.985244 15595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:22.985427 15571 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:22.985512 15571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "95875a4556fa43c5b39f640233a9fe13"
format_stamp: "Formatted at 2026-08-08 23:02:21 on dist-test-slave-f0tn"
server_key: "d88f02659256f77b77348ffabc26534a"
server_key_iv: "a80481bd6259776befd898c9fd4a0e63"
server_key_version: "encryptionkey@0"
I20260808 23:02:22.985821 15571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:23.003564 15571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:23.005862 15571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:23.006009 15571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:23.006243 15571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:02:23.006543 15571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:02:23.006578 15571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:23.006608 15571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:02:23.006631 15571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:23.012378 15571 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:43727
I20260808 23:02:23.012440 15757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:43727 every 8 connection(s)
I20260808 23:02:23.012763 15571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 23:02:23.013784 15761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:23.013428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.3:35779)
0808 23:02:23.013563 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:23.013642 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":71,"threads_started":1}
W20260808 23:02:23.014143 15758 heartbeater.cc:644] Failed to heartbeat to 127.23.194.62:34697 (0 consecutive failures): Network error: Failed to ping master at 127.23.194.62:34697: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
I20260808 23:02:23.014259 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 15571
I20260808 23:02:23.014451 15761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:23.014341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.3:43471)
0808 23:02:23.014385 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:23.014422 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 23:02:23.014853 15761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:23.014759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.3:59935)
0808 23:02:23.014804 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:23.014825 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 23:02:23.014963 15758 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 23:02:23.856333 15765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:23.856100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.1:58225)
0808 23:02:23.856234 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:23.856269 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260808 23:02:23.881055 15767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:23.880886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.2:45637)
0808 23:02:23.880980 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:23.881009 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":52,"threads_started":1}
I20260808 23:02:24.015429 15768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:24.015237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.3:38491)
0808 23:02:24.015346 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:24.015378 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260808 23:02:24.507035 15770 raft_consensus.cc:482] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:02:24.507165 15770 raft_consensus.cc:504] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } }
I20260808 23:02:24.507463 15770 leader_election.cc:288] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 95875a4556fa43c5b39f640233a9fe13 (127.23.194.3:43727), 686c6bae2d95420bb3911980f6dde11d (127.23.194.2:43531)
Aug 08 23:02:24 dist-test-slave-f0tn krb5kdc[13389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230142, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.2@KRBTEST.COM
Aug 08 23:02:24 dist-test-slave-f0tn krb5kdc[13389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230142, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.3@KRBTEST.COM
I20260808 23:02:24.519583 15774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:24.507801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:41617 (local address 127.23.194.2:43531)
0808 23:02:24.507932 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:24.507941 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:24.507960 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:02:24.508378 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:24.508380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:24.508418 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:24.508541 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:24.509370 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:24.510464 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:24.511968 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:24.512582 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:24.512686 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:24.515498 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:24.515503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:24.515503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:24.515515 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:24.517677 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:24.518216 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:24.518219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:24.518220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:24.518330 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:24.518643 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:24.518644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:24.518644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:24.518932 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:24.519006 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:24.519351 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 23:02:24.519473 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":39,"threads_started":1}
I20260808 23:02:24.519606 15771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:24.507617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:43727 (local address 127.23.194.1:52879)
0808 23:02:24.507741 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:24.507750 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:24.507800 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:24.508617 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:24.508620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:24.509067 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:24.509254 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:24.509261 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:24.510475 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:24.510476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:24.511658 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:24.511662 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:24.512845 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:24.512846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:24.512937 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:24.513629 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:24.513634 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:24.515655 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:24.517834 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:24.517835 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:24.517835 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:24.518166 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:24.518442 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:24.518442 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:24.518443 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:24.518548 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:24.519027 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:24.519035 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:24.519183 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 23:02:24.519271 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 23:02:24.519458 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":55,"thread_start_us":68,"threads_started":1}
I20260808 23:02:24.519872 15772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:24.507620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:43531 (local address 127.23.194.1:41617)
0808 23:02:24.507814 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:24.507822 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:24.507864 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:24.508617 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:24.508620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:24.509070 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:24.509254 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:24.509261 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:24.510537 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:24.510538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:24.511882 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:24.511885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:24.512787 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:24.512788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:24.512910 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:24.513459 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:24.513469 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:24.515339 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:24.517770 (+ 2431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:24.517777 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:24.517779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:24.518123 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:24.518387 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:24.518388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:24.518388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:24.518550 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:24.519046 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:24.519047 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:24.519185 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 23:02:24.519285 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 23:02:24.519540 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":129,"thread_start_us":48,"threads_started":1}
I20260808 23:02:24.519942 15773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:24.507734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:52879 (local address 127.23.194.3:43727)
0808 23:02:24.507889 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:24.507901 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:24.507935 (+ 34us) server_negotiation.cc:407] Connection header received
0808 23:02:24.508369 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:24.508371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:24.508438 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:24.508692 (+ 254us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:24.509398 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:24.510357 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:24.511768 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:24.512715 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:24.513224 (+ 509us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:24.515778 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:24.515782 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:24.515783 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:24.515795 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:24.517759 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:24.518259 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:24.518263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:24.518263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:24.518375 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:24.518611 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:24.518612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:24.518612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:24.518931 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:24.519620 (+ 689us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:24.519702 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 23:02:24.519836 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":44,"threads_started":1}
W20260808 23:02:24.520830 15214 leader_election.cc:341] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 95875a4556fa43c5b39f640233a9fe13 (127.23.194.3:43727): Not found: Tablet not found: d42573bd05f24bfbbaae40595e619be0
W20260808 23:02:24.521031 15211 leader_election.cc:341] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 686c6bae2d95420bb3911980f6dde11d (127.23.194.2:43531): Not found: Tablet not found: d42573bd05f24bfbbaae40595e619be0
I20260808 23:02:24.521106 15211 leader_election.cc:302] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1228fa7b2dab4969ad74438a5495e98f; no voters: 686c6bae2d95420bb3911980f6dde11d, 95875a4556fa43c5b39f640233a9fe13
I20260808 23:02:24.521237 15770 raft_consensus.cc:2726] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 23:02:24.856866 15772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:24.856739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.1:43157)
0808 23:02:24.856789 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:24.856825 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 23:02:24.881650 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:24.881455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.2:46289)
0808 23:02:24.881564 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:24.881607 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":59,"threads_started":1}
I20260808 23:02:25.016068 15777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:25.015851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.3:53173)
0808 23:02:25.015971 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:25.016012 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260808 23:02:25.857584 15778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:25.857324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.1:58387)
0808 23:02:25.857476 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:25.857522 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260808 23:02:25.882310 15779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:25.882091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.2:50575)
0808 23:02:25.882224 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:25.882267 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260808 23:02:26.016693 15780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:26.016493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.3:58459)
0808 23:02:26.016611 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:26.016650 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260808 23:02:26.603364 15782 raft_consensus.cc:482] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:02:26.603467 15782 raft_consensus.cc:504] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } }
I20260808 23:02:26.603613 15782 leader_election.cc:288] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 95875a4556fa43c5b39f640233a9fe13 (127.23.194.3:43727), 686c6bae2d95420bb3911980f6dde11d (127.23.194.2:43531)
W20260808 23:02:26.604008 15214 leader_election.cc:341] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 95875a4556fa43c5b39f640233a9fe13 (127.23.194.3:43727): Not found: Tablet not found: d42573bd05f24bfbbaae40595e619be0
W20260808 23:02:26.604135 15211 leader_election.cc:341] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 686c6bae2d95420bb3911980f6dde11d (127.23.194.2:43531): Not found: Tablet not found: d42573bd05f24bfbbaae40595e619be0
I20260808 23:02:26.604208 15211 leader_election.cc:302] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1228fa7b2dab4969ad74438a5495e98f; no voters: 686c6bae2d95420bb3911980f6dde11d, 95875a4556fa43c5b39f640233a9fe13
I20260808 23:02:26.604278 15782 raft_consensus.cc:2726] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 23:02:26.858196 15783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:26.857973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.1:45329)
0808 23:02:26.858082 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:26.858115 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":61,"threads_started":1}
I20260808 23:02:26.882903 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:26.882708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.2:56853)
0808 23:02:26.882823 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:26.882855 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
I20260808 23:02:27.017303 15785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:27.017082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.3:35183)
0808 23:02:27.017209 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:27.017246 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":76,"threads_started":1}
I20260808 23:02:27.858807 15786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:27.858613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.1:36549)
0808 23:02:27.858726 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:27.858760 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":62,"threads_started":1}
I20260808 23:02:27.883441 15787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:27.883268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.2:39737)
0808 23:02:27.883375 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:27.883405 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":55,"threads_started":1}
I20260808 23:02:28.017922 15788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:28.017712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.3:45429)
0808 23:02:28.017840 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:28.017879 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":61,"threads_started":1}
I20260808 23:02:28.859411 15791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:28.859193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.1:43011)
0808 23:02:28.859327 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:28.859363 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
I20260808 23:02:28.883977 15793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:28.883792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.2:35145)
0808 23:02:28.883908 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:28.883938 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260808 23:02:29.018481 15794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:29.018299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.3:49363)
0808 23:02:29.018406 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:29.018438 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":62,"threads_started":1}
I20260808 23:02:29.633807 15795 raft_consensus.cc:482] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:02:29.633909 15795 raft_consensus.cc:504] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } }
I20260808 23:02:29.634070 15795 leader_election.cc:288] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 95875a4556fa43c5b39f640233a9fe13 (127.23.194.3:43727), 686c6bae2d95420bb3911980f6dde11d (127.23.194.2:43531)
W20260808 23:02:29.634452 15214 leader_election.cc:341] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 95875a4556fa43c5b39f640233a9fe13 (127.23.194.3:43727): Not found: Tablet not found: d42573bd05f24bfbbaae40595e619be0
W20260808 23:02:29.634557 15211 leader_election.cc:341] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 686c6bae2d95420bb3911980f6dde11d (127.23.194.2:43531): Not found: Tablet not found: d42573bd05f24bfbbaae40595e619be0
I20260808 23:02:29.634595 15211 leader_election.cc:302] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1228fa7b2dab4969ad74438a5495e98f; no voters: 686c6bae2d95420bb3911980f6dde11d, 95875a4556fa43c5b39f640233a9fe13
I20260808 23:02:29.634661 15795 raft_consensus.cc:2726] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 23:02:29.860030 15796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:29.859827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.1:36479)
0808 23:02:29.859940 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:29.859978 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
I20260808 23:02:29.884521 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:29.884322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.2:51571)
0808 23:02:29.884438 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:29.884467 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":70,"threads_started":1}
I20260808 23:02:30.019074 15800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:30.018855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.3:35357)
0808 23:02:30.018985 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:30.019024 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":67,"threads_started":1}
I20260808 23:02:30.860692 15801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:30.860466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.1:58125)
0808 23:02:30.860597 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:30.860639 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260808 23:02:30.885210 15802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:30.884989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.2:35617)
0808 23:02:30.885126 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:30.885161 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260808 23:02:31.019712 15803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:31.019492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.3:55345)
0808 23:02:31.019619 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:31.019669 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
I20260808 23:02:31.861454 15805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:31.861189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.1:46147)
0808 23:02:31.861334 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:31.861406 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
I20260808 23:02:31.885865 15806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:31.885674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.2:38709)
0808 23:02:31.885786 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:31.885827 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":57,"threads_started":1}
I20260808 23:02:32.020414 15807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:32.020152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.3:58717)
0808 23:02:32.020320 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:32.020367 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":67,"threads_started":1}
I20260808 23:02:32.862275 15811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:32.862020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.1:59627)
0808 23:02:32.862180 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:32.862227 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":90,"threads_started":1}
I20260808 23:02:32.886620 15812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:32.886326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.2:34425)
0808 23:02:32.886511 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:32.886565 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":72,"threads_started":1}
I20260808 23:02:33.021148 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:33.020901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.3:36521)
0808 23:02:33.021057 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:33.021103 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
I20260808 23:02:33.862917 15814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:33.862689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.1:51269)
0808 23:02:33.862832 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:33.862870 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":85,"threads_started":1}
I20260808 23:02:33.887306 15815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:33.887089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.2:39981)
0808 23:02:33.887236 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:33.887266 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260808 23:02:34.021814 15817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:34.021584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.3:57531)
0808 23:02:34.021716 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:34.021769 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
I20260808 23:02:34.506942 15820 raft_consensus.cc:482] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:02:34.507045 15820 raft_consensus.cc:504] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95875a4556fa43c5b39f640233a9fe13" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43727 } } peers { permanent_uuid: "686c6bae2d95420bb3911980f6dde11d" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43531 } } peers { permanent_uuid: "1228fa7b2dab4969ad74438a5495e98f" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34835 } }
I20260808 23:02:34.507220 15820 leader_election.cc:288] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 95875a4556fa43c5b39f640233a9fe13 (127.23.194.3:43727), 686c6bae2d95420bb3911980f6dde11d (127.23.194.2:43531)
W20260808 23:02:34.507591 15211 leader_election.cc:341] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 686c6bae2d95420bb3911980f6dde11d (127.23.194.2:43531): Not found: Tablet not found: d42573bd05f24bfbbaae40595e619be0
W20260808 23:02:34.507705 15214 leader_election.cc:341] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 95875a4556fa43c5b39f640233a9fe13 (127.23.194.3:43727): Not found: Tablet not found: d42573bd05f24bfbbaae40595e619be0
I20260808 23:02:34.507767 15214 leader_election.cc:302] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1228fa7b2dab4969ad74438a5495e98f; no voters: 686c6bae2d95420bb3911980f6dde11d, 95875a4556fa43c5b39f640233a9fe13
I20260808 23:02:34.507836 15820 raft_consensus.cc:2726] T d42573bd05f24bfbbaae40595e619be0 P 1228fa7b2dab4969ad74438a5495e98f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 23:02:34.863523 15822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:34.863330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.1:56835)
0808 23:02:34.863444 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:34.863476 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
I20260808 23:02:34.887902 15823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:34.887699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.2:44041)
0808 23:02:34.887827 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:34.887863 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
I20260808 23:02:35.022400 15825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:35.022198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.3:35505)
0808 23:02:35.022316 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:35.022356 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260808 23:02:35.864137 15826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:35.863924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.1:58917)
0808 23:02:35.864049 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:35.864080 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260808 23:02:35.888535 15827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:35.888313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.2:35297)
0808 23:02:35.888455 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:35.888488 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":63,"threads_started":1}
I20260808 23:02:36.023164 15828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:36.022845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.3:51999)
0808 23:02:36.023064 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:36.023112 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":84,"threads_started":1}
I20260808 23:02:36.864898 15829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:36.864579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.1:58023)
0808 23:02:36.864803 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:36.864848 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":96,"threads_started":1}
I20260808 23:02:36.889147 15831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:36.888936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.2:36721)
0808 23:02:36.889061 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:36.889102 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260808 23:02:37.023826 15832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:37.023607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.3:51585)
0808 23:02:37.023741 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:37.023781 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
I20260808 23:02:37.865633 15833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:37.865434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.1:53479)
0808 23:02:37.865547 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:37.865584 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":64,"threads_started":1}
I20260808 23:02:37.889791 15835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:37.889607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.2:41307)
0808 23:02:37.889719 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:37.889747 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260808 23:02:38.017640 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 15146
I20260808 23:02:38.024556 15836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:38.024348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34697 (local address 127.23.194.3:33855)
0808 23:02:38.024472 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:38.024512 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":65,"threads_started":1}
I20260808 23:02:38.028448 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 15376
I20260808 23:02:38.040843 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 15571
2026-08-08T23:02:38Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18352 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:02:38 dist-test-slave-f0tn krb5kdc[15839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:02:38 dist-test-slave-f0tn krb5kdc[15839](info): set up 2 sockets
Aug 08 23:02:38 dist-test-slave-f0tn krb5kdc[15839](info): commencing operation
krb5kdc: starting...
W20260808 23:02:39.195699 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:02:39 dist-test-slave-f0tn krb5kdc[15839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:02:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:02:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:02:39.311833 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:40997
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:39431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:40997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:02:39.393821 15858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:39.394071 15858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:39.394104 15858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:39.395231 15858 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 23:02:39.395269 15858 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 23:02:39.395280 15858 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 23:02:39.395290 15858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:02:39.395303 15858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:39.395313 15858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:39.395327 15858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:02:39.395341 15858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:02:39.396816 15858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:39431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:40997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:40997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:39.397028 15858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:39.397200 15858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:39.401983 15866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:39.402014 15864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:39.401983 15868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:39.402009 15858 server_base.cc:1034] running on GCE node
I20260808 23:02:39.402581 15858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:39.402853 15858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:39.404004 15858 hybrid_clock.cc:648] HybridClock initialized: now 1786230159403993 us; error 34 us; skew 500 ppm
Aug 08 23:02:39 dist-test-slave-f0tn krb5kdc[15839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230159, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:39.406440 15858 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:02:39.406881 15858 webserver.cc:460] Webserver started at http://127.23.194.62:33287/ using document root <none> and password file <none>
I20260808 23:02:39.407090 15858 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:39.407137 15858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:39.407261 15858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:02:39.408262 15858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b"
format_stamp: "Formatted at 2026-08-08 23:02:39 on dist-test-slave-f0tn"
server_key: "d538a82747e7b4e42c62cb329c0144c9"
server_key_iv: "2a4319ce934ced1970e2770c5e4fcedb"
server_key_version: "encryptionkey@0"
I20260808 23:02:39.408591 15858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b"
format_stamp: "Formatted at 2026-08-08 23:02:39 on dist-test-slave-f0tn"
server_key: "d538a82747e7b4e42c62cb329c0144c9"
server_key_iv: "2a4319ce934ced1970e2770c5e4fcedb"
server_key_version: "encryptionkey@0"
I20260808 23:02:39.409864 15858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 23:02:39.410530 15880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:39.410720 15858 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:39.410804 15858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b"
format_stamp: "Formatted at 2026-08-08 23:02:39 on dist-test-slave-f0tn"
server_key: "d538a82747e7b4e42c62cb329c0144c9"
server_key_iv: "2a4319ce934ced1970e2770c5e4fcedb"
server_key_version: "encryptionkey@0"
I20260808 23:02:39.410861 15858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:39.422641 15858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:39.424471 15858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:39.424616 15858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:39.428550 15858 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:40997
I20260808 23:02:39.428592 15958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:40997 every 8 connection(s)
I20260808 23:02:39.428925 15858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:02:39.429826 15959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:02:39.432034 15959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b: Bootstrap starting.
I20260808 23:02:39.432577 15959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b: Neither blocks nor log segments found. Creating new log.
I20260808 23:02:39.432824 15959 log.cc:824] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:39.433508 15959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b: No bootstrap required, opened a new log
I20260808 23:02:39.435081 15959 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40997 } }
I20260808 23:02:39.435257 15959 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:02:39.435282 15959 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b9f5ac76f484b99a03e9d1cdeb8067b, State: Initialized, Role: FOLLOWER
I20260808 23:02:39.435388 15959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40997 } }
I20260808 23:02:39.435446 15959 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:02:39.435477 15959 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:02:39.435518 15959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:02:39.436077 15959 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40997 } }
I20260808 23:02:39.436178 15959 leader_election.cc:302] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b9f5ac76f484b99a03e9d1cdeb8067b; no voters:
I20260808 23:02:39.436352 15959 leader_election.cc:288] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:02:39.436403 15962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:02:39.436573 15962 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 1 LEADER]: Becoming Leader. State: Replica: 3b9f5ac76f484b99a03e9d1cdeb8067b, State: Running, Role: LEADER
I20260808 23:02:39.436791 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 15858
I20260808 23:02:39.436887 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:02:39.437062 24328 external_mini_cluster.cc:1428] Setting key ff12820d6dcd9ece0648e118b62b6ee3
I20260808 23:02:39.436602 15959 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:02:39.436703 15962 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40997 } }
I20260808 23:02:39.438338 15964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b9f5ac76f484b99a03e9d1cdeb8067b. Latest consensus state: current_term: 1 leader_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40997 } } }
I20260808 23:02:39.438444 15964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [sys.catalog]: This master's current role is: LEADER
I20260808 23:02:39.438334 15963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40997 } } }
I20260808 23:02:39.438755 15963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [sys.catalog]: This master's current role is: LEADER
I20260808 23:02:39.438851 15975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:02:39.439349 15975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:02:39.440879 15975 catalog_manager.cc:1347] Generated new cluster ID: f384cdec21564a56aa84ff73cd751acd
I20260808 23:02:39.440927 15975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 23:02:39 dist-test-slave-f0tn krb5kdc[15839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:39.447245 15966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:39.437543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44120 (local address 127.23.194.62:40997)
0808 23:02:39.438377 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:39.438388 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:39.438448 (+ 60us) server_negotiation.cc:407] Connection header received
0808 23:02:39.438576 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:39.438583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:39.438641 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:39.439702 (+ 1061us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:39.439786 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:39.440785 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:39.441665 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:39.442085 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:39.442179 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:39.444366 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:39.444370 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:39.444370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:39.444381 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:39.445866 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:39.446183 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:39.446186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:39.446186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:39.446283 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:39.446459 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:39.446460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:39.446460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:39.446688 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:39.446834 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:39.446926 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 23:02:39.447059 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":72,"threads_started":1}
I20260808 23:02:39.464946 15975 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:02:39.465557 15975 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:02:39.471278 15975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b: Generated new TSK 0
I20260808 23:02:39.471462 15975 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
I20260808 23:02:39.500321 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:40997
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:39431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:02:39.604171 15992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:39.604400 15992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:39.604427 15992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:39.605756 15992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:39.605810 15992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:39.605844 15992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:02:39.607537 15992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:39431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:40997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:39.607779 15992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:39.608009 15992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:39.613135 16003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:39.613165 15999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:39.613194 15992 server_base.cc:1034] running on GCE node
W20260808 23:02:39.613165 16001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:39.613669 15992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:39.613907 15992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:39.615090 15992 hybrid_clock.cc:648] HybridClock initialized: now 1786230159615077 us; error 48 us; skew 500 ppm
Aug 08 23:02:39 dist-test-slave-f0tn krb5kdc[15839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230159, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:39.617657 15992 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 23:02:39.618127 15992 webserver.cc:460] Webserver started at http://127.23.194.1:37173/ using document root <none> and password file <none>
I20260808 23:02:39.618367 15992 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:39.618417 15992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:39.618541 15992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:02:39.619576 15992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "b48d7f8f597c49a1a998e304a3d36019"
format_stamp: "Formatted at 2026-08-08 23:02:39 on dist-test-slave-f0tn"
server_key: "6b5abe57553bcc71b67097698c041f1f"
server_key_iv: "bfc3396832466c9a8b086372a1b9d8c7"
server_key_version: "encryptionkey@0"
I20260808 23:02:39.619911 15992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "b48d7f8f597c49a1a998e304a3d36019"
format_stamp: "Formatted at 2026-08-08 23:02:39 on dist-test-slave-f0tn"
server_key: "6b5abe57553bcc71b67097698c041f1f"
server_key_iv: "bfc3396832466c9a8b086372a1b9d8c7"
server_key_version: "encryptionkey@0"
I20260808 23:02:39.621198 15992 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 23:02:39.621992 16014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:39.622143 15992 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:39.622239 15992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "b48d7f8f597c49a1a998e304a3d36019"
format_stamp: "Formatted at 2026-08-08 23:02:39 on dist-test-slave-f0tn"
server_key: "6b5abe57553bcc71b67097698c041f1f"
server_key_iv: "bfc3396832466c9a8b086372a1b9d8c7"
server_key_version: "encryptionkey@0"
I20260808 23:02:39.622315 15992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:39.629050 15992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:39.630911 15992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:39.631049 15992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:39.631285 15992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:02:39.631572 15992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:02:39.631606 15992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:39.631637 15992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:02:39.631661 15992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:39.637198 15992 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:36185
I20260808 23:02:39.637261 16171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:36185 every 8 connection(s)
I20260808 23:02:39.637677 15992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
Aug 08 23:02:39 dist-test-slave-f0tn krb5kdc[15839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230159, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:39.646535 15966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:39.638443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:38603 (local address 127.23.194.62:40997)
0808 23:02:39.638491 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:39.638493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:39.638768 (+ 275us) server_negotiation.cc:407] Connection header received
0808 23:02:39.639281 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:39.639282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:39.639300 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:39.639350 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:39.640300 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:39.640811 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:39.641856 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:39.642060 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:39.642116 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:39.644074 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:39.644076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:39.644076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:39.644084 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:39.645218 (+ 1134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:39.645633 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:39.645634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:39.645635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:39.645706 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:39.645928 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:39.645929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:39.645929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:39.646040 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:39.646098 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:39.646360 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 23:02:39.646446 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 23:02:39.646574 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 15992
I20260808 23:02:39.646682 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:02:39.646693 16176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:39.638447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.1:38603)
0808 23:02:39.638630 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:39.638679 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:39.639175 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:39.639440 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:39.639445 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:39.640000 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:39.640199 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:39.640204 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:39.640897 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:39.640898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:39.641772 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:39.641774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:39.642131 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:39.642131 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:39.642255 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:39.642678 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:39.642686 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:39.643983 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:39.645292 (+ 1309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:39.645299 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:39.645301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:39.645567 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:39.645764 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:39.645764 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:39.645765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:39.645873 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:39.646095 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:39.646102 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:39.646242 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 23:02:39.646329 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 23:02:39.646483 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":99,"threads_started":1}
I20260808 23:02:39.646888 24328 external_mini_cluster.cc:1428] Setting key 4170947d7f11e65b9c5abd43a62e3535
I20260808 23:02:39.647192 16172 heartbeater.cc:347] Connected to a master server at 127.23.194.62:40997
I20260808 23:02:39.647277 16172 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:39.647444 16172 heartbeater.cc:511] Master 127.23.194.62:40997 requested a full tablet report, sending...
I20260808 23:02:39.647966 15906 ts_manager.cc:194] Registered new tserver with Master: b48d7f8f597c49a1a998e304a3d36019 (127.23.194.1:36185)
I20260808 23:02:39.648582 15906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:38603
WARNING: no policy specified for kudu/127.23.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
I20260808 23:02:39.672168 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:40997
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:39431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:02:39.752259 16180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:39.752457 16180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:39.752478 16180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:39.753571 16180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:39.753602 16180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:39.753626 16180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:02:39.755009 16180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:39431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:40997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:39.755198 16180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:39.755373 16180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:39.760401 16189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:39.760473 16180 server_base.cc:1034] running on GCE node
W20260808 23:02:39.760407 16193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:39.760407 16187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:39.760807 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:39.761015 16180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:39.762146 16180 hybrid_clock.cc:648] HybridClock initialized: now 1786230159762129 us; error 29 us; skew 500 ppm
Aug 08 23:02:39 dist-test-slave-f0tn krb5kdc[15839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230159, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:39.764428 16180 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 23:02:39.764827 16180 webserver.cc:460] Webserver started at http://127.23.194.2:34769/ using document root <none> and password file <none>
I20260808 23:02:39.764999 16180 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:39.765036 16180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:39.765126 16180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:02:39.766175 16180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "2092826ce0b442eda24e730644f09c57"
format_stamp: "Formatted at 2026-08-08 23:02:39 on dist-test-slave-f0tn"
server_key: "71853a32ec7f1eda357c7ade005c1d0d"
server_key_iv: "b39ae36bb899d3c23736c5c6929214b2"
server_key_version: "encryptionkey@0"
I20260808 23:02:39.766465 16180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "2092826ce0b442eda24e730644f09c57"
format_stamp: "Formatted at 2026-08-08 23:02:39 on dist-test-slave-f0tn"
server_key: "71853a32ec7f1eda357c7ade005c1d0d"
server_key_iv: "b39ae36bb899d3c23736c5c6929214b2"
server_key_version: "encryptionkey@0"
I20260808 23:02:39.767614 16180 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 23:02:39.768234 16200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:39.768380 16180 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:02:39.768455 16180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "2092826ce0b442eda24e730644f09c57"
format_stamp: "Formatted at 2026-08-08 23:02:39 on dist-test-slave-f0tn"
server_key: "71853a32ec7f1eda357c7ade005c1d0d"
server_key_iv: "b39ae36bb899d3c23736c5c6929214b2"
server_key_version: "encryptionkey@0"
I20260808 23:02:39.768517 16180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:39.783381 16180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:39.785533 16180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:39.785665 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:39.785872 16180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:02:39.786177 16180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:02:39.786211 16180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:39.786232 16180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:02:39.786244 16180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:39.791625 16180 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:43831
I20260808 23:02:39.791688 16364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:43831 every 8 connection(s)
I20260808 23:02:39.791992 16180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:02:39.797315 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 16180
I20260808 23:02:39.797443 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:02:39.797636 24328 external_mini_cluster.cc:1428] Setting key 5baf1018c65534f01f5650f42a763727
Aug 08 23:02:39 dist-test-slave-f0tn krb5kdc[15839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230159, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:39.800601 15966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:39.792657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:39189 (local address 127.23.194.62:40997)
0808 23:02:39.792703 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:39.792705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:39.792924 (+ 219us) server_negotiation.cc:407] Connection header received
0808 23:02:39.793430 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:39.793431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:39.793440 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:39.793483 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:39.794418 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:39.794917 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:39.795874 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:39.796072 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:39.796125 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:39.798101 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:39.798103 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:39.798103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:39.798111 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:39.799382 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:39.799742 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:39.799743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:39.799744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:39.799800 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:39.800011 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:39.800012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:39.800012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:39.800123 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:39.800180 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:39.800426 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 23:02:39.800512 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 23:02:39.800756 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:39.792630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.2:39189)
0808 23:02:39.792800 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:39.792842 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:39.793312 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:39.793548 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:39.793551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:39.794124 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:39.794349 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:39.794354 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:39.794991 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:39.794992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:39.795792 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:39.795794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:39.796132 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:39.796133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:39.796245 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:39.796648 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:39.796657 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:39.798012 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:39.799457 (+ 1445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:39.799463 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:39.799466 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:39.799678 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:39.799854 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:39.799854 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:39.799855 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:39.799957 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:39.800177 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:39.800184 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:39.800313 (+ 129us) client_negotiation.cc:769] Sending connection context
0808 23:02:39.800399 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 23:02:39.800546 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260808 23:02:39.801144 16365 heartbeater.cc:347] Connected to a master server at 127.23.194.62:40997
I20260808 23:02:39.801230 16365 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:39.801427 16365 heartbeater.cc:511] Master 127.23.194.62:40997 requested a full tablet report, sending...
I20260808 23:02:39.801815 15906 ts_manager.cc:194] Registered new tserver with Master: 2092826ce0b442eda24e730644f09c57 (127.23.194.2:43831)
I20260808 23:02:39.802299 15906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:39189
WARNING: no policy specified for kudu/127.23.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
I20260808 23:02:39.822145 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:40997
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:39431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:02:39.904336 16372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:02:39.904539 16372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:02:39.904561 16372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:02:39.905725 16372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:02:39.905766 16372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:02:39.905790 16372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:02:39.907363 16372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:39431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:40997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:02:39.907603 16372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:02:39.907810 16372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:02:39.912822 16384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:02:39.912997 16380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:39.912874 16372 server_base.cc:1034] running on GCE node
W20260808 23:02:39.912822 16381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:02:39.913305 16372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:02:39.913617 16372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:02:39.914757 16372 hybrid_clock.cc:648] HybridClock initialized: now 1786230159914741 us; error 26 us; skew 500 ppm
Aug 08 23:02:39 dist-test-slave-f0tn krb5kdc[15839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230159, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:02:39.917224 16372 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 23:02:39.917754 16372 webserver.cc:460] Webserver started at http://127.23.194.3:42681/ using document root <none> and password file <none>
I20260808 23:02:39.917989 16372 fs_manager.cc:359] Metadata directory not provided
I20260808 23:02:39.918044 16372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:02:39.918179 16372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:02:39.919229 16372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "2087427f68454ae4920271588cd4af76"
format_stamp: "Formatted at 2026-08-08 23:02:39 on dist-test-slave-f0tn"
server_key: "94d061ae0b191e08dccc2b329577e80d"
server_key_iv: "a568eaa0be339e78d446faaa1f4b5095"
server_key_version: "encryptionkey@0"
I20260808 23:02:39.919593 16372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "2087427f68454ae4920271588cd4af76"
format_stamp: "Formatted at 2026-08-08 23:02:39 on dist-test-slave-f0tn"
server_key: "94d061ae0b191e08dccc2b329577e80d"
server_key_iv: "a568eaa0be339e78d446faaa1f4b5095"
server_key_version: "encryptionkey@0"
I20260808 23:02:39.920958 16372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:02:39.921705 16395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:39.921854 16372 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:39.921947 16372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "2087427f68454ae4920271588cd4af76"
format_stamp: "Formatted at 2026-08-08 23:02:39 on dist-test-slave-f0tn"
server_key: "94d061ae0b191e08dccc2b329577e80d"
server_key_iv: "a568eaa0be339e78d446faaa1f4b5095"
server_key_version: "encryptionkey@0"
I20260808 23:02:39.922014 16372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:02:39.966215 16372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:02:39.968369 16372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:02:39.968497 16372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:02:39.968693 16372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:02:39.969063 16372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:02:39.969110 16372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:39.969135 16372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:02:39.969149 16372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:02:39.975713 16372 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:36141
I20260808 23:02:39.975781 16551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:36141 every 8 connection(s)
I20260808 23:02:39.976104 16372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 23:02:39.978314 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 16372
I20260808 23:02:39.978403 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:02:39.978571 24328 external_mini_cluster.cc:1428] Setting key befa4b8421333422f6e60118bf5dc227
Aug 08 23:02:39 dist-test-slave-f0tn krb5kdc[15839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230159, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:02:39.985042 15966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:39.976853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:32777 (local address 127.23.194.62:40997)
0808 23:02:39.976912 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:39.976914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:39.977107 (+ 193us) server_negotiation.cc:407] Connection header received
0808 23:02:39.977660 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:39.977662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:39.977674 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:39.977724 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:39.978890 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:39.979389 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:39.980350 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:39.980550 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:39.980605 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:39.982684 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:39.982685 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:39.982686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:39.982693 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:39.983787 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:39.984161 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:39.984162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:39.984162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:39.984216 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:39.984437 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:39.984438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:39.984439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:39.984549 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:39.984605 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:39.984870 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 23:02:39.984952 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 23:02:39.985205 16556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:39.976816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.3:32777)
0808 23:02:39.976992 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:39.977029 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:39.977566 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:39.977804 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:39.977809 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:39.978505 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:02:39.978800 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:39.978806 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:39.979471 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:39.979472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:39.980271 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:39.980273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:39.980618 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:39.980618 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:39.980762 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:39.981242 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:02:39.981250 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:02:39.982600 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:02:39.983859 (+ 1259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:39.983867 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:39.983869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:39.984079 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:39.984272 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:02:39.984273 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:02:39.984273 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:02:39.984381 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:02:39.984604 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:02:39.984611 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:02:39.984753 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 23:02:39.984841 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 23:02:39.984995 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260808 23:02:39.985664 16552 heartbeater.cc:347] Connected to a master server at 127.23.194.62:40997
I20260808 23:02:39.985766 16552 heartbeater.cc:464] Registering TS with master...
I20260808 23:02:39.985947 16552 heartbeater.cc:511] Master 127.23.194.62:40997 requested a full tablet report, sending...
I20260808 23:02:39.986323 15906 ts_manager.cc:194] Registered new tserver with Master: 2087427f68454ae4920271588cd4af76 (127.23.194.3:36141)
I20260808 23:02:39.986660 15906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:32777
I20260808 23:02:39.990274 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:02:39.997316 15966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:39.991601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44128 (local address 127.23.194.62:40997)
0808 23:02:39.991662 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:39.991664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:39.991755 (+ 91us) server_negotiation.cc:407] Connection header received
0808 23:02:39.991827 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:39.991828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:39.991836 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:39.991878 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:02:39.992435 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:39.992902 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:39.993722 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:39.993902 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:39.993951 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:39.995119 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:02:39.995120 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:02:39.995120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:02:39.995126 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:02:39.996332 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:39.996671 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:39.996672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:39.996672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:39.996716 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:02:39.996849 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:02:39.996849 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:02:39.996850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:02:39.996972 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:02:39.997040 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:39.997202 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 23:02:39.997256 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 23:02:39.998709 15906 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44128:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:02:40.000342 15906 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:02:40.006916 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:40.002623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:36185 (local address 127.0.0.1:56452)
0808 23:02:40.002783 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:40.002823 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:40.002887 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:40.003528 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:40.003529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:40.003530 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:02:40.003567 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:40.003568 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.004828 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.004830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:40.005750 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:40.005753 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.006636 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.006636 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:40.006704 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:40.006715 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 23:02:40.006781 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 23:02:40.006834 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":5,"thread_start_us":70,"threads_started":1}
I20260808 23:02:40.006930 16572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:40.002754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56452 (local address 127.23.194.1:36185)
0808 23:02:40.003388 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:40.003392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:40.003401 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:02:40.003437 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:40.003438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:40.003452 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:40.003534 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:02:40.003616 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.004696 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.005830 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.006558 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.006639 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:40.006708 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:40.006805 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 23:02:40.006850 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":102,"threads_started":1}
I20260808 23:02:40.007107 16576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:40.002997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33678 (local address 127.23.194.3:36141)
0808 23:02:40.003151 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:40.003156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:40.003166 (+ 10us) server_negotiation.cc:407] Connection header received
0808 23:02:40.003209 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:40.003211 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:40.003228 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:40.003312 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:02:40.003483 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.004650 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.006010 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.006701 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.006768 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:40.006825 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:40.006974 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 23:02:40.007022 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260808 23:02:40.007181 16569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:40.002632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:36141 (local address 127.0.0.1:33678)
0808 23:02:40.002950 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:40.002961 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:40.003015 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:40.003334 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:40.003338 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:40.003340 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:02:40.003410 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:40.003412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.004766 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.004767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:40.005928 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:40.005930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.006773 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.006774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:40.006840 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:40.006844 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:02:40.006891 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 23:02:40.006924 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":1,"thread_start_us":95,"threads_started":1}
I20260808 23:02:40.008010 16462 tablet_service.cc:1467] Processing CreateTablet for tablet 2a8fc450c4bb4b6c89b7d912e639feed (DEFAULT_TABLE table=test-table [id=eefce6bf49624068aad5bed61b77a49a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:02:40.008311 16574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:40.002845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:43831 (local address 127.0.0.1:54016)
0808 23:02:40.003182 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:40.003191 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:40.003245 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:40.003734 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:40.003735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:40.003736 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:02:40.003777 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:40.003779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.004618 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.004619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:40.005922 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:40.005925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.008063 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.008064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:40.008126 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:40.008131 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:02:40.008203 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 23:02:40.008245 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":8,"thread_start_us":76,"threads_started":1}
I20260808 23:02:40.008412 16462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a8fc450c4bb4b6c89b7d912e639feed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:02:40.008347 16079 tablet_service.cc:1467] Processing CreateTablet for tablet 2a8fc450c4bb4b6c89b7d912e639feed (DEFAULT_TABLE table=test-table [id=eefce6bf49624068aad5bed61b77a49a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:02:40.008577 16575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:40.002957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54016 (local address 127.23.194.2:43831)
0808 23:02:40.003122 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:40.003128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:40.003237 (+ 109us) server_negotiation.cc:407] Connection header received
0808 23:02:40.003626 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:40.003630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:40.003658 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:40.003733 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:02:40.003831 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.004519 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.006866 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.007986 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.008053 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:40.008110 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:40.008458 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 23:02:40.008504 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260808 23:02:40.008692 16079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a8fc450c4bb4b6c89b7d912e639feed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:02:40.009181 16269 tablet_service.cc:1467] Processing CreateTablet for tablet 2a8fc450c4bb4b6c89b7d912e639feed (DEFAULT_TABLE table=test-table [id=eefce6bf49624068aad5bed61b77a49a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:02:40.009548 16269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a8fc450c4bb4b6c89b7d912e639feed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:02:40.011117 16577 tablet_bootstrap.cc:492] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76: Bootstrap starting.
I20260808 23:02:40.011646 16577 tablet_bootstrap.cc:654] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76: Neither blocks nor log segments found. Creating new log.
I20260808 23:02:40.012121 16578 tablet_bootstrap.cc:492] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019: Bootstrap starting.
I20260808 23:02:40.012285 16577 log.cc:824] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:40.012779 16578 tablet_bootstrap.cc:654] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019: Neither blocks nor log segments found. Creating new log.
I20260808 23:02:40.012784 16580 tablet_bootstrap.cc:492] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57: Bootstrap starting.
I20260808 23:02:40.013361 16578 log.cc:824] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:40.013460 16577 tablet_bootstrap.cc:492] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76: No bootstrap required, opened a new log
I20260808 23:02:40.013469 16580 tablet_bootstrap.cc:654] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57: Neither blocks nor log segments found. Creating new log.
I20260808 23:02:40.013531 16577 ts_tablet_manager.cc:1397] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 23:02:40.013998 16580 log.cc:824] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57: Log is configured to *not* fsync() on all Append() calls
I20260808 23:02:40.014191 16578 tablet_bootstrap.cc:492] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019: No bootstrap required, opened a new log
I20260808 23:02:40.014256 16578 ts_tablet_manager.cc:1397] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 23:02:40.014842 16580 tablet_bootstrap.cc:492] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57: No bootstrap required, opened a new log
I20260808 23:02:40.014936 16580 ts_tablet_manager.cc:1397] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 23:02:40.015285 16577 raft_consensus.cc:348] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } }
I20260808 23:02:40.015429 16577 raft_consensus.cc:374] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:02:40.015462 16577 raft_consensus.cc:729] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2087427f68454ae4920271588cd4af76, State: Initialized, Role: FOLLOWER
I20260808 23:02:40.015595 16577 consensus_queue.cc:260] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } }
I20260808 23:02:40.015887 16552 heartbeater.cc:503] Master 127.23.194.62:40997 was elected leader, sending a full tablet report...
I20260808 23:02:40.016227 16577 ts_tablet_manager.cc:1428] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 23:02:40.016147 16578 raft_consensus.cc:348] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } }
I20260808 23:02:40.016309 16578 raft_consensus.cc:374] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:02:40.016340 16578 raft_consensus.cc:729] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b48d7f8f597c49a1a998e304a3d36019, State: Initialized, Role: FOLLOWER
I20260808 23:02:40.016448 16578 consensus_queue.cc:260] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } }
I20260808 23:02:40.016724 16578 ts_tablet_manager.cc:1428] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 23:02:40.016860 16172 heartbeater.cc:503] Master 127.23.194.62:40997 was elected leader, sending a full tablet report...
I20260808 23:02:40.016830 16580 raft_consensus.cc:348] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } }
I20260808 23:02:40.017001 16580 raft_consensus.cc:374] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:02:40.017050 16580 raft_consensus.cc:729] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2092826ce0b442eda24e730644f09c57, State: Initialized, Role: FOLLOWER
I20260808 23:02:40.017174 16580 consensus_queue.cc:260] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } }
I20260808 23:02:40.017486 16580 ts_tablet_manager.cc:1428] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260808 23:02:40.017529 16365 heartbeater.cc:503] Master 127.23.194.62:40997 was elected leader, sending a full tablet report...
W20260808 23:02:40.042572 16366 tablet.cc:2369] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:02:40.055619 16587 raft_consensus.cc:482] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:02:40.055766 16587 raft_consensus.cc:504] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } }
I20260808 23:02:40.056131 16587 leader_election.cc:288] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2092826ce0b442eda24e730644f09c57 (127.23.194.2:43831), 2087427f68454ae4920271588cd4af76 (127.23.194.3:36141)
I20260808 23:02:40.058801 16176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:40.056315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:43831 (local address 127.23.194.1:35743)
0808 23:02:40.056392 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:40.056404 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:40.056454 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:40.056573 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:40.056574 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:40.056576 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:02:40.056644 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:40.056646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.057264 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.057265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:40.058037 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:40.058039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.058563 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.058564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:40.058636 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:40.058639 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:02:40.058686 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 23:02:40.058726 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 23:02:40.058801 16575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:40.056429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:35743 (local address 127.23.194.2:43831)
0808 23:02:40.056472 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:40.056474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:40.056482 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:02:40.056507 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:40.056508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:40.056517 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:40.056565 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:02:40.056713 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.057194 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.058122 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.058495 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.058559 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:40.058569 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:40.058695 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 23:02:40.058732 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 23:02:40.059087 16576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:40.056405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:39793 (local address 127.23.194.3:36141)
0808 23:02:40.056454 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:40.056456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:40.056617 (+ 161us) server_negotiation.cc:407] Connection header received
0808 23:02:40.056673 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:40.056674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:40.056685 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:40.056743 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:02:40.056890 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.057360 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.058323 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.058712 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.058784 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:40.058793 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:40.058974 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 23:02:40.059023 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 23:02:40.059170 16306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a8fc450c4bb4b6c89b7d912e639feed" candidate_uuid: "b48d7f8f597c49a1a998e304a3d36019" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2092826ce0b442eda24e730644f09c57" is_pre_election: true
I20260808 23:02:40.059333 16306 raft_consensus.cc:2454] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b48d7f8f597c49a1a998e304a3d36019 in term 0.
I20260808 23:02:40.059351 16589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:40.056334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:36141 (local address 127.23.194.1:39793)
0808 23:02:40.056537 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:40.056546 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:02:40.056611 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:02:40.056749 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:02:40.056750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:02:40.056752 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:02:40.056827 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:40.056829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.057471 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.057472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:40.058225 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:02:40.058227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.058794 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.058796 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:02:40.058867 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:40.058872 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:02:40.059215 (+ 343us) client_negotiation.cc:240] Negotiation successful
0808 23:02:40.059263 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":136,"threads_started":1}
I20260808 23:02:40.059576 16022 leader_election.cc:302] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2092826ce0b442eda24e730644f09c57, b48d7f8f597c49a1a998e304a3d36019; no voters:
I20260808 23:02:40.059723 16587 raft_consensus.cc:2781] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:02:40.059774 16587 raft_consensus.cc:482] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:02:40.059790 16587 raft_consensus.cc:3037] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:02:40.059748 16489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a8fc450c4bb4b6c89b7d912e639feed" candidate_uuid: "b48d7f8f597c49a1a998e304a3d36019" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2087427f68454ae4920271588cd4af76" is_pre_election: true
I20260808 23:02:40.059909 16489 raft_consensus.cc:2454] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b48d7f8f597c49a1a998e304a3d36019 in term 0.
I20260808 23:02:40.060509 16587 raft_consensus.cc:504] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } }
I20260808 23:02:40.060638 16587 leader_election.cc:288] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [CANDIDATE]: Term 1 election: Requested vote from peers 2092826ce0b442eda24e730644f09c57 (127.23.194.2:43831), 2087427f68454ae4920271588cd4af76 (127.23.194.3:36141)
I20260808 23:02:40.060806 16489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a8fc450c4bb4b6c89b7d912e639feed" candidate_uuid: "b48d7f8f597c49a1a998e304a3d36019" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2087427f68454ae4920271588cd4af76"
I20260808 23:02:40.060822 16306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a8fc450c4bb4b6c89b7d912e639feed" candidate_uuid: "b48d7f8f597c49a1a998e304a3d36019" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2092826ce0b442eda24e730644f09c57"
I20260808 23:02:40.060885 16306 raft_consensus.cc:3037] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:02:40.060885 16489 raft_consensus.cc:3037] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:02:40.061604 16306 raft_consensus.cc:2454] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b48d7f8f597c49a1a998e304a3d36019 in term 1.
I20260808 23:02:40.061700 16489 raft_consensus.cc:2454] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b48d7f8f597c49a1a998e304a3d36019 in term 1.
I20260808 23:02:40.061810 16022 leader_election.cc:302] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2092826ce0b442eda24e730644f09c57, b48d7f8f597c49a1a998e304a3d36019; no voters:
I20260808 23:02:40.061893 16587 raft_consensus.cc:2781] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:02:40.062043 16587 raft_consensus.cc:686] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [term 1 LEADER]: Becoming Leader. State: Replica: b48d7f8f597c49a1a998e304a3d36019, State: Running, Role: LEADER
I20260808 23:02:40.062143 16587 consensus_queue.cc:237] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } }
I20260808 23:02:40.063050 15905 catalog_manager.cc:5495] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 reported cstate change: term changed from 0 to 1, leader changed from <none> to b48d7f8f597c49a1a998e304a3d36019 (127.23.194.1). New cstate: current_term: 1 leader_uuid: "b48d7f8f597c49a1a998e304a3d36019" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } health_report { overall_health: UNKNOWN } } }
I20260808 23:02:40.081090 16572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:40.079073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56458 (local address 127.23.194.1:36185)
0808 23:02:40.079126 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 23:02:40.079128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:02:40.079136 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:02:40.079175 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:02:40.079175 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:02:40.079194 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:02:40.079251 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:02:40.079355 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.079820 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.080492 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:02:40.080680 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:02:40.080724 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:02:40.080830 (+ 106us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:02:40.080931 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:02:40.080985 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 23:02:40.080997 (+ 12us) server_negotiation.cc:299] Negotiation successful
0808 23:02:40.081013 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 23:02:40.082854 16306 raft_consensus.cc:1261] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57 [term 1 FOLLOWER]: Refusing update from remote peer b48d7f8f597c49a1a998e304a3d36019: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:02:40.082854 16489 raft_consensus.cc:1261] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76 [term 1 FOLLOWER]: Refusing update from remote peer b48d7f8f597c49a1a998e304a3d36019: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:02:40.083176 16590 consensus_queue.cc:1035] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:02:40.083312 16587 consensus_queue.cc:1035] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:02:40.085438 16593 mvcc.cc:205] Tried to move back new op lower bound from 7316398735696846848 to 7316398735615262720. Current Snapshot: MvccSnapshot[applied={T|T < 7316398735696846848}]
I20260808 23:02:40.086226 16594 mvcc.cc:205] Tried to move back new op lower bound from 7316398735696846848 to 7316398735615262720. Current Snapshot: MvccSnapshot[applied={T|T < 7316398735696846848 or (T in {7316398735696846848})}]
I20260808 23:02:40.087692 15905 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:02:40.087798 15905 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:02:40.088589 15905 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b: Sending DeleteTablet for 3 replicas of tablet 2a8fc450c4bb4b6c89b7d912e639feed
I20260808 23:02:40.088913 16079 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a8fc450c4bb4b6c89b7d912e639feed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:02:40 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:56452
I20260808 23:02:40.088922 16269 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a8fc450c4bb4b6c89b7d912e639feed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:02:40 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:54016
I20260808 23:02:40.088941 16462 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a8fc450c4bb4b6c89b7d912e639feed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:02:40 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:33678
I20260808 23:02:40.089148 16611 tablet_replica.cc:330] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76: stopping tablet replica
I20260808 23:02:40.089210 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 15992
I20260808 23:02:40.089243 16611 raft_consensus.cc:2229] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:02:40.094089 16611 raft_consensus.cc:2258] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:02:40.094967 16612 tablet_replica.cc:330] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57: stopping tablet replica
I20260808 23:02:40.095100 16612 raft_consensus.cc:2229] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:02:40.095230 16612 raft_consensus.cc:2258] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:02:40.096060 16612 ts_tablet_manager.cc:1905] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:02:40.096709 16611 ts_tablet_manager.cc:1905] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:02:40.097640 16612 ts_tablet_manager.cc:1918] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:02:40.097716 16612 log.cc:1196] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/2a8fc450c4bb4b6c89b7d912e639feed
I20260808 23:02:40.097971 16612 ts_tablet_manager.cc:1939] T 2a8fc450c4bb4b6c89b7d912e639feed P 2092826ce0b442eda24e730644f09c57: Deleting consensus metadata
I20260808 23:02:40.098232 16611 ts_tablet_manager.cc:1918] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:02:40.098292 16611 log.cc:1196] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/2a8fc450c4bb4b6c89b7d912e639feed
I20260808 23:02:40.098412 15883 catalog_manager.cc:4841] TS 2092826ce0b442eda24e730644f09c57 (127.23.194.2:43831): tablet 2a8fc450c4bb4b6c89b7d912e639feed (table test-table [id=eefce6bf49624068aad5bed61b77a49a]) successfully deleted
I20260808 23:02:40.098537 16611 ts_tablet_manager.cc:1939] T 2a8fc450c4bb4b6c89b7d912e639feed P 2087427f68454ae4920271588cd4af76: Deleting consensus metadata
I20260808 23:02:40.099295 15881 catalog_manager.cc:4841] TS 2087427f68454ae4920271588cd4af76 (127.23.194.3:36141): tablet 2a8fc450c4bb4b6c89b7d912e639feed (table test-table [id=eefce6bf49624068aad5bed61b77a49a]) successfully deleted
W20260808 23:02:40.106943 15883 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.194.1:36185 (error 108)
I20260808 23:02:40.107129 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 16180
I20260808 23:02:40.109015 16574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:02:40.108880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:36185 (local address 127.0.0.1:56474)
0808 23:02:40.108937 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 23:02:40.108979 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.1:36185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 23:02:40.110004 15883 catalog_manager.cc:4584] TS b48d7f8f597c49a1a998e304a3d36019 (127.23.194.1:36185): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2a8fc450c4bb4b6c89b7d912e639feed: Network error: Client connection negotiation failed: client connection to 127.23.194.1:36185: connect: Connection refused (error 111)
I20260808 23:02:40.125005 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 16372
I20260808 23:02:40.133500 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 15858
I20260808 23:03:00.144857 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:40997
--webserver_interface=127.23.194.62
--webserver_port=33287
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:39431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:40997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:00.226323 16614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:00.226526 16614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:00.226549 16614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:00.227711 16614 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 23:03:00.227751 16614 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 23:03:00.227762 16614 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 23:03:00.227772 16614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:03:00.227785 16614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:00.227814 16614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:00.227838 16614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:03:00.227860 16614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:03:00.229331 16614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:39431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:40997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:40997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=33287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:00.229574 16614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:00.229796 16614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:00.239144 16628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:00.239126 16625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:00.239248 16614 server_base.cc:1034] running on GCE node
W20260808 23:03:00.239164 16624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:00.239678 16614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:00.239974 16614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:00.241122 16614 hybrid_clock.cc:648] HybridClock initialized: now 1786230180241101 us; error 41 us; skew 500 ppm
Aug 08 23:03:00 dist-test-slave-f0tn krb5kdc[15839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230180, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:00.246790 16614 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:03:00.247288 16614 webserver.cc:460] Webserver started at http://127.23.194.62:33287/ using document root <none> and password file <none>
I20260808 23:03:00.247494 16614 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:00.247552 16614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:00.248939 16614 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:03:00.249639 16637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:00.249809 16614 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:00.249898 16614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b"
format_stamp: "Formatted at 2026-08-08 23:02:39 on dist-test-slave-f0tn"
server_key: "d538a82747e7b4e42c62cb329c0144c9"
server_key_iv: "2a4319ce934ced1970e2770c5e4fcedb"
server_key_version: "encryptionkey@0"
I20260808 23:03:00.250313 16614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:00.266279 16614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:00.268028 16614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:00.268134 16614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:00.272153 16614 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:40997
I20260808 23:03:00.272219 16715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:40997 every 8 connection(s)
I20260808 23:03:00.272647 16614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:03:00.274282 16717 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:03:00.274989 16717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b: Bootstrap starting.
I20260808 23:03:00.280671 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 16614
I20260808 23:03:00.280939 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:36185
--local_ip_for_outbound_sockets=127.23.194.1
--tserver_master_addrs=127.23.194.62:40997
--webserver_port=37173
--webserver_interface=127.23.194.1
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:39431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:03:00.284335 16717 log.cc:824] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:00.286296 16717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:03:00.286574 16717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b: Bootstrap complete.
I20260808 23:03:00.288728 16717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40997 } }
I20260808 23:03:00.288955 16717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b9f5ac76f484b99a03e9d1cdeb8067b, State: Initialized, Role: FOLLOWER
I20260808 23:03:00.289106 16717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40997 } }
I20260808 23:03:00.289186 16717 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:03:00.289242 16717 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:03:00.289290 16717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 1 FOLLOWER]: Advancing to term 2
I20260808 23:03:00.290354 16717 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40997 } }
I20260808 23:03:00.290465 16717 leader_election.cc:302] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b9f5ac76f484b99a03e9d1cdeb8067b; no voters:
I20260808 23:03:00.290663 16717 leader_election.cc:288] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 23:03:00.290705 16722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 2 FOLLOWER]: Leader election won for term 2
I20260808 23:03:00.290825 16722 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 2 LEADER]: Becoming Leader. State: Replica: 3b9f5ac76f484b99a03e9d1cdeb8067b, State: Running, Role: LEADER
I20260808 23:03:00.290918 16722 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40997 } }
I20260808 23:03:00.290956 16717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:03:00.291231 16722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b9f5ac76f484b99a03e9d1cdeb8067b. Latest consensus state: current_term: 2 leader_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40997 } } }
I20260808 23:03:00.291342 16722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:00.291234 16724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40997 } } }
I20260808 23:03:00.291538 16724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:00.291643 16731 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:03:00.292469 16731 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:03:00.292569 16731 catalog_manager.cc:1259] Loaded cluster ID: f384cdec21564a56aa84ff73cd751acd
I20260808 23:03:00.292819 16731 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:03:00.294075 16731 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:03:00.294261 16731 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b: Loaded TSK: 0
I20260808 23:03:00.310025 16731 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b: Generated new TSK 1
I20260808 23:03:00.310117 16731 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 23:03:00.368626 16720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:00.368822 16720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:00.368844 16720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:00.370008 16720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:00.370052 16720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:00.370077 16720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:03:00.371551 16720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:39431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:36185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=37173
--tserver_master_addrs=127.23.194.62:40997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:00.371765 16720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:00.371977 16720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:00.377760 16761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:00.377816 16720 server_base.cc:1034] running on GCE node
W20260808 23:03:00.377764 16763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:00.377830 16760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:00.378167 16720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:00.378450 16720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:00.379624 16720 hybrid_clock.cc:648] HybridClock initialized: now 1786230180379605 us; error 35 us; skew 500 ppm
Aug 08 23:03:00 dist-test-slave-f0tn krb5kdc[15839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230180, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:00.382606 16720 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 23:03:00.383107 16720 webserver.cc:460] Webserver started at http://127.23.194.1:37173/ using document root <none> and password file <none>
I20260808 23:03:00.383373 16720 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:00.383448 16720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:00.385109 16720 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:03:00.385903 16773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:00.386059 16720 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:00.386153 16720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "b48d7f8f597c49a1a998e304a3d36019"
format_stamp: "Formatted at 2026-08-08 23:02:39 on dist-test-slave-f0tn"
server_key: "6b5abe57553bcc71b67097698c041f1f"
server_key_iv: "bfc3396832466c9a8b086372a1b9d8c7"
server_key_version: "encryptionkey@0"
I20260808 23:03:00.386480 16720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:00.394614 16720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:00.396546 16720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:00.396657 16720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:00.396854 16720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:00.397265 16782 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 23:03:00.399245 16720 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 23:03:00.399292 16720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 23:03:00.399322 16720 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 23:03:00.399915 16720 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 23:03:00.399947 16720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 23:03:00.400046 16782 tablet_bootstrap.cc:492] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019: Bootstrap starting.
I20260808 23:03:00.406538 16720 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:36185
I20260808 23:03:00.406878 16720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:03:00.410667 16782 log.cc:824] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:00.410996 16943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:36185 every 8 connection(s)
I20260808 23:03:00.413268 16782 tablet_bootstrap.cc:492] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:03:00.413591 16782 tablet_bootstrap.cc:492] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019: Bootstrap complete.
I20260808 23:03:00.414705 16782 ts_tablet_manager.cc:1397] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.005s
Aug 08 23:03:00 dist-test-slave-f0tn krb5kdc[15839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230180, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:03:00.415998 16782 raft_consensus.cc:348] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } }
I20260808 23:03:00.416322 16782 raft_consensus.cc:729] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b48d7f8f597c49a1a998e304a3d36019, State: Initialized, Role: FOLLOWER
I20260808 23:03:00.416482 16782 consensus_queue.cc:260] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } }
I20260808 23:03:00.416810 16782 ts_tablet_manager.cc:1428] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260808 23:03:00.417034 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 16720
I20260808 23:03:00.417337 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:43831
--local_ip_for_outbound_sockets=127.23.194.2
--tserver_master_addrs=127.23.194.62:40997
--webserver_port=34769
--webserver_interface=127.23.194.2
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:39431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:03:00.419824 16948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:00.407542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.1:48615)
0808 23:03:00.409553 (+ 2011us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:00.409595 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:00.410073 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:00.410582 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:00.410586 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:00.410919 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:00.411187 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:00.411195 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.412325 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.412326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:00.413111 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:00.413113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.413805 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.413807 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:00.413932 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:00.414414 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:03:00.414422 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:00.415779 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:00.418101 (+ 2322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:00.418109 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:00.418111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:00.418391 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:00.418661 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:00.418662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:00.418662 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:00.418783 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:00.419245 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:00.419251 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:00.419367 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 23:03:00.419454 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 23:03:00.419622 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1970,"thread_start_us":98,"threads_started":1}
I20260808 23:03:00.419824 16950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:00.407646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:48615 (local address 127.23.194.62:40997)
0808 23:03:00.410246 (+ 2600us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:00.410253 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:00.410281 (+ 28us) server_negotiation.cc:407] Connection header received
0808 23:03:00.410381 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:00.410385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:00.410423 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:00.410535 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:00.411299 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.412205 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.413200 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.413706 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.413807 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:00.415876 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:00.415880 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:00.415880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:00.415892 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:00.418011 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:00.418481 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:00.418484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:00.418485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:00.418591 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:00.418845 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:00.418846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:00.418846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:00.419129 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:00.419218 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:00.419522 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 23:03:00.419653 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2550,"thread_start_us":76,"threads_started":1}
I20260808 23:03:00.420594 16944 heartbeater.cc:347] Connected to a master server at 127.23.194.62:40997
I20260808 23:03:00.420687 16944 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:00.420849 16944 heartbeater.cc:511] Master 127.23.194.62:40997 requested a full tablet report, sending...
I20260808 23:03:00.421566 16661 ts_manager.cc:194] Registered new tserver with Master: b48d7f8f597c49a1a998e304a3d36019 (127.23.194.1:36185)
I20260808 23:03:00.422626 16661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:48615
I20260808 23:03:00.423417 16944 heartbeater.cc:503] Master 127.23.194.62:40997 was elected leader, sending a full tablet report...
I20260808 23:03:00.428290 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:00.422129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:36185 (local address 127.0.0.1:37810)
0808 23:03:00.425480 (+ 3351us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:00.425510 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:00.425579 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:00.425737 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:00.425739 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:00.425742 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:00.425807 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:00.425809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.426598 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.426599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:00.427402 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:00.427405 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.428002 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.428003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:00.428086 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:00.428098 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 23:03:00.428162 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 23:03:00.428220 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3321,"thread_start_us":95,"threads_started":1}
I20260808 23:03:00.428292 16956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:00.422384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37810 (local address 127.23.194.1:36185)
0808 23:03:00.422538 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:00.422542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:00.425573 (+ 3031us) server_negotiation.cc:407] Connection header received
0808 23:03:00.425653 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:00.425655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:00.425671 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:00.425736 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:00.425870 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.426523 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.427480 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.427929 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.427993 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:00.428047 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:00.428180 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 23:03:00.428219 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
I20260808 23:03:00.428987 16833 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a8fc450c4bb4b6c89b7d912e639feed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:02:40 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:37810
I20260808 23:03:00.429481 16959 tablet_replica.cc:330] stopping tablet replica
I20260808 23:03:00.429569 16959 raft_consensus.cc:2229] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:03:00.429634 16959 raft_consensus.cc:2258] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:03:00.429925 16959 ts_tablet_manager.cc:1905] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:03:00.431834 16959 ts_tablet_manager.cc:1918] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:03:00.431893 16959 log.cc:1196] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/wal/wals/2a8fc450c4bb4b6c89b7d912e639feed
I20260808 23:03:00.432091 16959 ts_tablet_manager.cc:1939] T 2a8fc450c4bb4b6c89b7d912e639feed P b48d7f8f597c49a1a998e304a3d36019: Deleting consensus metadata
I20260808 23:03:00.432564 16641 catalog_manager.cc:4841] TS b48d7f8f597c49a1a998e304a3d36019 (127.23.194.1:36185): tablet 2a8fc450c4bb4b6c89b7d912e639feed (table test-table [id=eefce6bf49624068aad5bed61b77a49a]) successfully deleted
W20260808 23:03:00.501350 16954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:00.501583 16954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:00.501614 16954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:00.502753 16954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:00.502791 16954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:00.502825 16954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:03:00.504256 16954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:39431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:43831
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=34769
--tserver_master_addrs=127.23.194.62:40997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:00.504485 16954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:00.504689 16954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:00.509550 16967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:00.509601 16954 server_base.cc:1034] running on GCE node
W20260808 23:03:00.509577 16970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:00.509577 16968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:00.509927 16954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:00.510176 16954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:00.511319 16954 hybrid_clock.cc:648] HybridClock initialized: now 1786230180511308 us; error 31 us; skew 500 ppm
Aug 08 23:03:00 dist-test-slave-f0tn krb5kdc[15839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230180, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:00.514516 16954 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 23:03:00.515027 16954 webserver.cc:460] Webserver started at http://127.23.194.2:34769/ using document root <none> and password file <none>
I20260808 23:03:00.515316 16954 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:00.515400 16954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:00.517174 16954 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:03:00.518106 16978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:00.518270 16954 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 23:03:00.518369 16954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "2092826ce0b442eda24e730644f09c57"
format_stamp: "Formatted at 2026-08-08 23:02:39 on dist-test-slave-f0tn"
server_key: "71853a32ec7f1eda357c7ade005c1d0d"
server_key_iv: "b39ae36bb899d3c23736c5c6929214b2"
server_key_version: "encryptionkey@0"
I20260808 23:03:00.518759 16954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:00.537453 16954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:00.539753 16954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:00.539911 16954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:00.540154 16954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:00.540506 16954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:00.540555 16954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:00.540587 16954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:00.540616 16954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:00.546902 16954 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:43831
I20260808 23:03:00.546958 17143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:43831 every 8 connection(s)
I20260808 23:03:00.547309 16954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
Aug 08 23:03:00 dist-test-slave-f0tn krb5kdc[15839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230180, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:03:00.554526 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 16954
I20260808 23:03:00.554785 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:36141
--local_ip_for_outbound_sockets=127.23.194.3
--tserver_master_addrs=127.23.194.62:40997
--webserver_port=42681
--webserver_interface=127.23.194.3
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:39431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:03:00.556406 16950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:00.548079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:48231 (local address 127.23.194.62:40997)
0808 23:03:00.548124 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:00.548126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:00.548317 (+ 191us) server_negotiation.cc:407] Connection header received
0808 23:03:00.548781 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:00.548782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:00.548796 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:00.548857 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:00.549793 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.550321 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.551286 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.551486 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.551549 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:00.553560 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:00.553562 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:00.553562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:00.553571 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:00.555054 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:00.555503 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:00.555504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:00.555504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:00.555563 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:00.555803 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:00.555803 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:00.555804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:00.555902 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:00.555952 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:00.556258 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 23:03:00.556338 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 23:03:00.556586 17147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:00.548057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.2:48231)
0808 23:03:00.548213 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:00.548246 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:00.548696 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:00.548909 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:00.548912 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:00.549512 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:00.549706 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:00.549711 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.550394 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.550394 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:00.551195 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:00.551197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.551554 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.551555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:00.551656 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:00.552071 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:03:00.552079 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:00.553465 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:00.555144 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:00.555151 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:00.555153 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:00.555423 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:00.555617 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:00.555617 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:00.555618 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:00.555745 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:00.555953 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:00.555958 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:00.556103 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 23:03:00.556212 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 23:03:00.556365 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
I20260808 23:03:00.557003 17144 heartbeater.cc:347] Connected to a master server at 127.23.194.62:40997
I20260808 23:03:00.557108 17144 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:00.557303 17144 heartbeater.cc:511] Master 127.23.194.62:40997 requested a full tablet report, sending...
I20260808 23:03:00.557698 16661 ts_manager.cc:194] Registered new tserver with Master: 2092826ce0b442eda24e730644f09c57 (127.23.194.2:43831)
I20260808 23:03:00.558046 16661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:48231
W20260808 23:03:00.639237 17148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:00.639451 17148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:00.639483 17148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:00.640650 17148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:00.640691 17148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:00.640725 17148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:03:00.642287 17148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:39431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:36141
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=42681
--tserver_master_addrs=127.23.194.62:40997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:00.642545 17148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:00.642755 17148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:00.647794 17154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:00.647877 17155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:00.647794 17158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:00.648022 17148 server_base.cc:1034] running on GCE node
I20260808 23:03:00.648228 17148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:00.648449 17148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:00.649583 17148 hybrid_clock.cc:648] HybridClock initialized: now 1786230180649579 us; error 46 us; skew 500 ppm
Aug 08 23:03:00 dist-test-slave-f0tn krb5kdc[15839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230180, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:00.652050 17148 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 23:03:00.652551 17148 webserver.cc:460] Webserver started at http://127.23.194.3:42681/ using document root <none> and password file <none>
I20260808 23:03:00.652806 17148 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:00.652877 17148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:00.654284 17148 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:03:00.654860 17165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:00.655028 17148 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:00.655110 17148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "2087427f68454ae4920271588cd4af76"
format_stamp: "Formatted at 2026-08-08 23:02:39 on dist-test-slave-f0tn"
server_key: "94d061ae0b191e08dccc2b329577e80d"
server_key_iv: "a568eaa0be339e78d446faaa1f4b5095"
server_key_version: "encryptionkey@0"
I20260808 23:03:00.655442 17148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:00.667785 17148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:00.669931 17148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:00.670050 17148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:00.670274 17148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:00.670539 17148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:00.670569 17148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:00.670591 17148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:00.670603 17148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:00.676137 17148 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:36141
I20260808 23:03:00.676218 17339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:36141 every 8 connection(s)
I20260808 23:03:00.676530 17148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 23:03:00.681138 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 17148
Aug 08 23:03:00 dist-test-slave-f0tn krb5kdc[15839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230180, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:03:00.685966 16950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:00.677266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:50267 (local address 127.23.194.62:40997)
0808 23:03:00.677327 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:00.677329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:00.677590 (+ 261us) server_negotiation.cc:407] Connection header received
0808 23:03:00.678097 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:00.678099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:00.678112 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:00.678178 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:00.679127 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.679623 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.680583 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.680777 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.680832 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:00.683241 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:00.683242 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:00.683243 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:00.683251 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:00.684576 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:00.684955 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:00.684957 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:00.684957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:00.685017 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:00.685235 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:00.685235 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:00.685236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:00.685358 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:00.685462 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:00.685793 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 23:03:00.685874 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 23:03:00.686161 17343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:00.677254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.3:50267)
0808 23:03:00.677466 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:00.677508 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:00.678000 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:00.678255 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:00.678259 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:00.678833 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:00.679028 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:00.679032 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.679714 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.679716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:00.680497 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:00.680500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.680847 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.680848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:00.680971 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:00.681504 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:03:00.681515 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:00.683149 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:00.684637 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:00.684645 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:00.684648 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:00.684880 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:00.685084 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:00.685084 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:00.685085 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:00.685180 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:00.685454 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:00.685460 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:00.685620 (+ 160us) client_negotiation.cc:769] Sending connection context
0808 23:03:00.685747 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 23:03:00.685893 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":89,"threads_started":1}
I20260808 23:03:00.686633 17340 heartbeater.cc:347] Connected to a master server at 127.23.194.62:40997
I20260808 23:03:00.686739 17340 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:00.686911 17340 heartbeater.cc:511] Master 127.23.194.62:40997 requested a full tablet report, sending...
I20260808 23:03:00.687299 16661 ts_manager.cc:194] Registered new tserver with Master: 2087427f68454ae4920271588cd4af76 (127.23.194.3:36141)
I20260808 23:03:00.687667 16661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:50267
I20260808 23:03:00.688237 17346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:00.681589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47526 (local address 127.23.194.62:40997)
0808 23:03:00.681700 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:00.681702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:00.681803 (+ 101us) server_negotiation.cc:407] Connection header received
0808 23:03:00.681877 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:00.681878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:00.681887 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:00.681938 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:00.682642 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.683168 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.683918 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.684140 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.684204 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:00.685451 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:00.685453 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:00.685453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:00.685462 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:00.687108 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:00.687522 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:00.687523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:00.687523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:00.687581 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:00.687784 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:00.687785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:00.687786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:00.687883 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:00.687996 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:00.688117 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 23:03:00.688177 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":72,"threads_started":1}
I20260808 23:03:00.688905 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:03:00.691592 17346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:00.689223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47532 (local address 127.23.194.62:40997)
0808 23:03:00.689262 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:00.689264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:00.689402 (+ 138us) server_negotiation.cc:407] Connection header received
0808 23:03:00.689459 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:00.689460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:00.689497 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:00.689553 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:03:00.689679 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.690116 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.690900 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.691106 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.691159 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:00.691320 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:03:00.691385 (+ 65us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 23:03:00.691537 (+ 152us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47532: token expired
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 23:03:00.691648 17346 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47532: token expired
W20260808 23:03:00.691844 16561 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.23.194.62:40997)
I20260808 23:03:00.697013 17346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:00.692068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47534 (local address 127.23.194.62:40997)
0808 23:03:00.692110 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:00.692111 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:00.692118 (+ 7us) server_negotiation.cc:407] Connection header received
0808 23:03:00.692168 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:00.692169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:00.692176 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:00.692218 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:00.692646 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.693069 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.693760 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.693927 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.693975 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:00.694944 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:00.694945 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:00.694946 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:00.694951 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:00.696093 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:00.696384 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:00.696385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:00.696385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:00.696425 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:00.696570 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:00.696570 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:00.696570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:00.696691 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:00.696760 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:00.696903 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 23:03:00.696957 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 23:03:00.698120 16661 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:03:00.699659 16661 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:03:00.701830 16833 tablet_service.cc:1467] Processing CreateTablet for tablet 28bc4374866a4fbb859e2a5135746331 (DEFAULT_TABLE table=test-table [id=cabe8a3e71e74d2c9d4d814b7f373928]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:00.702042 16833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28bc4374866a4fbb859e2a5135746331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:00.703502 16782 tablet_bootstrap.cc:492] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019: Bootstrap starting.
I20260808 23:03:00.703800 16782 tablet_bootstrap.cc:654] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:00.704291 16782 tablet_bootstrap.cc:492] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019: No bootstrap required, opened a new log
I20260808 23:03:00.704341 16782 ts_tablet_manager.cc:1397] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 23:03:00.704445 16782 raft_consensus.cc:348] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } }
I20260808 23:03:00.704512 16782 raft_consensus.cc:374] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:00.704559 16782 raft_consensus.cc:729] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b48d7f8f597c49a1a998e304a3d36019, State: Initialized, Role: FOLLOWER
I20260808 23:03:00.704603 16782 consensus_queue.cc:260] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } }
I20260808 23:03:00.704696 16782 ts_tablet_manager.cc:1428] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:00.705304 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:00.701343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:36141 (local address 127.0.0.1:57988)
0808 23:03:00.701429 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:00.701440 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:00.701479 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:00.701986 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:00.701988 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:00.701990 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:00.702027 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:00.702029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.703267 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.703268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:00.704276 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:00.704278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.705100 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.705100 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:00.705177 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:00.705180 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:03:00.705222 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 23:03:00.705257 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 23:03:00.705335 17350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:00.701624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57988 (local address 127.23.194.3:36141)
0808 23:03:00.701806 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:00.701810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:00.701821 (+ 11us) server_negotiation.cc:407] Connection header received
0808 23:03:00.701859 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:00.701861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:00.701877 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:00.702098 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:00.702198 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.703187 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.704356 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.705027 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.705101 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:00.705166 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:00.705246 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 23:03:00.705277 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":90,"threads_started":1}
I20260808 23:03:00.705760 17348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:00.701410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:43831 (local address 127.0.0.1:57510)
0808 23:03:00.701556 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:00.701563 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:00.701599 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:00.702053 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:00.702054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:00.702057 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:00.702096 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:00.702098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.703057 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.703058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:00.704215 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:00.704218 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.705572 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.705573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:00.705641 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:00.705645 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:03:00.705689 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 23:03:00.705716 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260808 23:03:00.705842 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:00.701513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57510 (local address 127.23.194.2:43831)
0808 23:03:00.701831 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:00.701835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:00.701843 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:03:00.701874 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:00.701875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:00.701887 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:00.701956 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:00.702183 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.702984 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.704304 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.705027 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.705101 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:00.705160 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:00.705724 (+ 564us) server_negotiation.cc:299] Negotiation successful
0808 23:03:00.705768 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":108,"threads_started":1}
I20260808 23:03:00.706027 17235 tablet_service.cc:1467] Processing CreateTablet for tablet 28bc4374866a4fbb859e2a5135746331 (DEFAULT_TABLE table=test-table [id=cabe8a3e71e74d2c9d4d814b7f373928]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:00.706333 17235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28bc4374866a4fbb859e2a5135746331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:00.706414 17045 tablet_service.cc:1467] Processing CreateTablet for tablet 28bc4374866a4fbb859e2a5135746331 (DEFAULT_TABLE table=test-table [id=cabe8a3e71e74d2c9d4d814b7f373928]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:00.706693 17045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28bc4374866a4fbb859e2a5135746331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:00.709057 17352 tablet_bootstrap.cc:492] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76: Bootstrap starting.
I20260808 23:03:00.709276 17353 tablet_bootstrap.cc:492] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57: Bootstrap starting.
I20260808 23:03:00.709661 17352 tablet_bootstrap.cc:654] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:00.709928 17353 tablet_bootstrap.cc:654] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:00.710212 17352 log.cc:824] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:00.710386 17353 log.cc:824] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:00.710933 17352 tablet_bootstrap.cc:492] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76: No bootstrap required, opened a new log
I20260808 23:03:00.711011 17352 ts_tablet_manager.cc:1397] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:03:00.711063 17353 tablet_bootstrap.cc:492] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57: No bootstrap required, opened a new log
I20260808 23:03:00.711126 17353 ts_tablet_manager.cc:1397] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:03:00.712929 17353 raft_consensus.cc:348] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } }
I20260808 23:03:00.712905 17352 raft_consensus.cc:348] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } }
I20260808 23:03:00.713083 17353 raft_consensus.cc:374] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:00.713093 17352 raft_consensus.cc:374] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:00.713126 17352 raft_consensus.cc:729] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2087427f68454ae4920271588cd4af76, State: Initialized, Role: FOLLOWER
I20260808 23:03:00.713121 17353 raft_consensus.cc:729] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2092826ce0b442eda24e730644f09c57, State: Initialized, Role: FOLLOWER
I20260808 23:03:00.713255 17352 consensus_queue.cc:260] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } }
I20260808 23:03:00.713255 17353 consensus_queue.cc:260] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } }
I20260808 23:03:00.713559 17352 ts_tablet_manager.cc:1428] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 23:03:00.713559 17353 ts_tablet_manager.cc:1428] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 23:03:00.713565 17144 heartbeater.cc:503] Master 127.23.194.62:40997 was elected leader, sending a full tablet report...
I20260808 23:03:00.713805 17340 heartbeater.cc:503] Master 127.23.194.62:40997 was elected leader, sending a full tablet report...
I20260808 23:03:00.729584 17358 raft_consensus.cc:482] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:03:00.729694 17358 raft_consensus.cc:504] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } }
I20260808 23:03:00.729982 17358 leader_election.cc:288] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2092826ce0b442eda24e730644f09c57 (127.23.194.2:43831), b48d7f8f597c49a1a998e304a3d36019 (127.23.194.1:36185)
I20260808 23:03:00.732630 17343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:00.730132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:43831 (local address 127.23.194.3:37431)
0808 23:03:00.730231 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:00.730243 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:00.730300 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:00.730465 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:00.730466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:00.730467 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:00.730528 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:00.730530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.731125 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.731126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:00.731893 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:00.731895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.732398 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.732398 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:00.732477 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:00.732480 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:03:00.732523 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 23:03:00.732561 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":37}
I20260808 23:03:00.732628 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:00.730180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:37431 (local address 127.23.194.2:43831)
0808 23:03:00.730235 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:00.730236 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:00.730292 (+ 56us) server_negotiation.cc:407] Connection header received
0808 23:03:00.730382 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:00.730383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:00.730394 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:00.730451 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:00.730584 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.731058 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.731967 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.732333 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.732392 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:00.732400 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:00.732533 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 23:03:00.732567 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 23:03:00.732738 16956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:00.730325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:48369 (local address 127.23.194.1:36185)
0808 23:03:00.730378 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:00.730380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:00.730478 (+ 98us) server_negotiation.cc:407] Connection header received
0808 23:03:00.730534 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:00.730535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:00.730545 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:00.730590 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:00.730729 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.731179 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.732065 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.732427 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.732487 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:00.732496 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:00.732634 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 23:03:00.732676 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 23:03:00.732746 17359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:00.730265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:36185 (local address 127.23.194.3:48369)
0808 23:03:00.730429 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:00.730437 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:00.730477 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:00.730600 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:00.730601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:00.730603 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:00.730672 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:00.730673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.731255 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.731256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:00.731989 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:00.731991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.732505 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.732506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:00.732577 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:00.732581 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:03:00.732623 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 23:03:00.732659 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":96,"threads_started":1}
I20260808 23:03:00.733016 17074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28bc4374866a4fbb859e2a5135746331" candidate_uuid: "2087427f68454ae4920271588cd4af76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2092826ce0b442eda24e730644f09c57" is_pre_election: true
I20260808 23:03:00.733177 17074 raft_consensus.cc:2454] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2087427f68454ae4920271588cd4af76 in term 0.
I20260808 23:03:00.733197 16876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28bc4374866a4fbb859e2a5135746331" candidate_uuid: "2087427f68454ae4920271588cd4af76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b48d7f8f597c49a1a998e304a3d36019" is_pre_election: true
I20260808 23:03:00.733347 16876 raft_consensus.cc:2454] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2087427f68454ae4920271588cd4af76 in term 0.
I20260808 23:03:00.733424 17169 leader_election.cc:302] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2087427f68454ae4920271588cd4af76, 2092826ce0b442eda24e730644f09c57; no voters:
I20260808 23:03:00.733543 17358 raft_consensus.cc:2781] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:03:00.733604 17358 raft_consensus.cc:482] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:03:00.733628 17358 raft_consensus.cc:3037] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:00.734295 17358 raft_consensus.cc:504] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } }
I20260808 23:03:00.734402 17358 leader_election.cc:288] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [CANDIDATE]: Term 1 election: Requested vote from peers 2092826ce0b442eda24e730644f09c57 (127.23.194.2:43831), b48d7f8f597c49a1a998e304a3d36019 (127.23.194.1:36185)
I20260808 23:03:00.734561 17074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28bc4374866a4fbb859e2a5135746331" candidate_uuid: "2087427f68454ae4920271588cd4af76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2092826ce0b442eda24e730644f09c57"
I20260808 23:03:00.734576 16876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28bc4374866a4fbb859e2a5135746331" candidate_uuid: "2087427f68454ae4920271588cd4af76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b48d7f8f597c49a1a998e304a3d36019"
I20260808 23:03:00.734642 17074 raft_consensus.cc:3037] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:00.734642 16876 raft_consensus.cc:3037] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:00.735464 16876 raft_consensus.cc:2454] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2087427f68454ae4920271588cd4af76 in term 1.
I20260808 23:03:00.735464 17074 raft_consensus.cc:2454] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2087427f68454ae4920271588cd4af76 in term 1.
I20260808 23:03:00.735625 17169 leader_election.cc:302] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2087427f68454ae4920271588cd4af76, 2092826ce0b442eda24e730644f09c57; no voters:
I20260808 23:03:00.735716 17358 raft_consensus.cc:2781] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:00.735855 17358 raft_consensus.cc:686] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [term 1 LEADER]: Becoming Leader. State: Replica: 2087427f68454ae4920271588cd4af76, State: Running, Role: LEADER
I20260808 23:03:00.735932 17358 consensus_queue.cc:237] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } }
I20260808 23:03:00.736726 16661 catalog_manager.cc:5495] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2087427f68454ae4920271588cd4af76 (127.23.194.3). New cstate: current_term: 1 leader_uuid: "2087427f68454ae4920271588cd4af76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } health_report { overall_health: UNKNOWN } } }
I20260808 23:03:00.750854 17350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:00.748646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58002 (local address 127.23.194.3:36141)
0808 23:03:00.748713 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:00.748714 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:00.748723 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:03:00.748759 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:00.748760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:00.748778 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:00.748836 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:03:00.748941 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.749423 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.750135 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:00.750324 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:00.750385 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:00.750534 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:03:00.750631 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:03:00.750706 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:00.750741 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 23:03:00.750760 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 23:03:00.752636 17074 raft_consensus.cc:1261] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57 [term 1 FOLLOWER]: Refusing update from remote peer 2087427f68454ae4920271588cd4af76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:03:00.752641 16876 raft_consensus.cc:1261] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Refusing update from remote peer 2087427f68454ae4920271588cd4af76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:03:00.752979 17361 consensus_queue.cc:1035] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:03:00.753103 17358 consensus_queue.cc:1035] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:03:00.755178 17366 mvcc.cc:205] Tried to move back new op lower bound from 7316398820360261632 to 7316398820295192576. Current Snapshot: MvccSnapshot[applied={T|T < 7316398820360261632}]
I20260808 23:03:00.755971 17369 mvcc.cc:205] Tried to move back new op lower bound from 7316398820360261632 to 7316398820295192576. Current Snapshot: MvccSnapshot[applied={T|T < 7316398820360261632}]
I20260808 23:03:00.756309 17367 mvcc.cc:205] Tried to move back new op lower bound from 7316398820360261632 to 7316398820295192576. Current Snapshot: MvccSnapshot[applied={T|T < 7316398820360261632 or (T in {7316398820360261632})}]
I20260808 23:03:00.757470 16660 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:03:00.757561 16660 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:03:00.758227 16660 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b: Sending DeleteTablet for 3 replicas of tablet 28bc4374866a4fbb859e2a5135746331
I20260808 23:03:00.758488 17235 tablet_service.cc:1514] Processing DeleteTablet for tablet 28bc4374866a4fbb859e2a5135746331 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:03:00 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:57988
I20260808 23:03:00.758514 16833 tablet_service.cc:1514] Processing DeleteTablet for tablet 28bc4374866a4fbb859e2a5135746331 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:03:00 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:37810
I20260808 23:03:00.758602 16959 tablet_replica.cc:330] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019: stopping tablet replica
I20260808 23:03:00.758643 16959 raft_consensus.cc:2229] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:03:00.758630 17045 tablet_service.cc:1514] Processing DeleteTablet for tablet 28bc4374866a4fbb859e2a5135746331 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:03:00 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:57510
I20260808 23:03:00.758697 16959 raft_consensus.cc:2258] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:03:00.758842 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 16720
I20260808 23:03:00.759018 16959 ts_tablet_manager.cc:1905] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:03:00.800125 17381 tablet_replica.cc:330] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57: stopping tablet replica
I20260808 23:03:00.800302 17381 raft_consensus.cc:2229] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:03:00.800431 17381 raft_consensus.cc:2258] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:03:00.800625 17378 tablet_replica.cc:330] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76: stopping tablet replica
I20260808 23:03:00.800761 17378 raft_consensus.cc:2229] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:03:00.800908 17378 raft_consensus.cc:2258] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:03:00.802989 17378 ts_tablet_manager.cc:1905] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:03:00.804756 17378 ts_tablet_manager.cc:1918] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:03:00.804831 17378 log.cc:1196] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/28bc4374866a4fbb859e2a5135746331
I20260808 23:03:00.805085 17378 ts_tablet_manager.cc:1939] T 28bc4374866a4fbb859e2a5135746331 P 2087427f68454ae4920271588cd4af76: Deleting consensus metadata
I20260808 23:03:00.805711 17381 ts_tablet_manager.cc:1905] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:03:00.805919 16638 catalog_manager.cc:4841] TS 2087427f68454ae4920271588cd4af76 (127.23.194.3:36141): tablet 28bc4374866a4fbb859e2a5135746331 (table test-table [id=cabe8a3e71e74d2c9d4d814b7f373928]) successfully deleted
I20260808 23:03:00.807413 17381 ts_tablet_manager.cc:1918] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:03:00.807508 17381 log.cc:1196] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/28bc4374866a4fbb859e2a5135746331
W20260808 23:03:00.807574 16641 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.194.1:36185 (error 108)
I20260808 23:03:00.808413 17381 ts_tablet_manager.cc:1939] T 28bc4374866a4fbb859e2a5135746331 P 2092826ce0b442eda24e730644f09c57: Deleting consensus metadata
W20260808 23:03:00.808858 17348 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:03:00.808166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:36185 (local address 127.0.0.1:37814)
0808 23:03:00.808254 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:00.808266 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:00.808341 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:00.808815 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.1:36185: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 23:03:00.809118 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 16954
W20260808 23:03:00.809087 16641 catalog_manager.cc:4584] TS b48d7f8f597c49a1a998e304a3d36019 (127.23.194.1:36185): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 28bc4374866a4fbb859e2a5135746331: Network error: Client connection negotiation failed: client connection to 127.23.194.1:36185: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 23:03:00.809630 16641 catalog_manager.cc:4841] TS 2092826ce0b442eda24e730644f09c57 (127.23.194.2:43831): tablet 28bc4374866a4fbb859e2a5135746331 (table test-table [id=cabe8a3e71e74d2c9d4d814b7f373928]) successfully deleted
I20260808 23:03:00.819969 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 17148
I20260808 23:03:00.828548 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 16614
I20260808 23:03:20.845920 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:40997
--webserver_interface=127.23.194.62
--webserver_port=33287
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:39431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:40997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:20.926569 17384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:20.926759 17384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:20.926779 17384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:20.927878 17384 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 23:03:20.927917 17384 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 23:03:20.927929 17384 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 23:03:20.927939 17384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:03:20.927960 17384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:20.927985 17384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:20.928007 17384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:03:20.928028 17384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:03:20.929481 17384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:39431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:40997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:40997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=33287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:20.929700 17384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:20.929903 17384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:20.934649 17392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:20.934672 17396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:20.934739 17394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:20.934837 17384 server_base.cc:1034] running on GCE node
I20260808 23:03:20.935060 17384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:20.935340 17384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:20.936476 17384 hybrid_clock.cc:648] HybridClock initialized: now 1786230200936465 us; error 29 us; skew 500 ppm
Aug 08 23:03:20 dist-test-slave-f0tn krb5kdc[15839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230200, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:20.943003 17384 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:03:20.943436 17384 webserver.cc:460] Webserver started at http://127.23.194.62:33287/ using document root <none> and password file <none>
I20260808 23:03:20.943612 17384 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:20.943652 17384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:20.944916 17384 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:03:20.945566 17409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:20.945751 17384 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 23:03:20.945933 17384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b"
format_stamp: "Formatted at 2026-08-08 23:02:39 on dist-test-slave-f0tn"
server_key: "d538a82747e7b4e42c62cb329c0144c9"
server_key_iv: "2a4319ce934ced1970e2770c5e4fcedb"
server_key_version: "encryptionkey@0"
I20260808 23:03:20.946301 17384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:20.963698 17384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:20.965442 17384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:20.965551 17384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:20.969231 17384 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:40997
I20260808 23:03:20.969266 17490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:40997 every 8 connection(s)
I20260808 23:03:20.969635 17384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:03:20.971029 17492 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:03:20.971372 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 17384
I20260808 23:03:20.971647 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:36185
--local_ip_for_outbound_sockets=127.23.194.1
--tserver_master_addrs=127.23.194.62:40997
--webserver_port=37173
--webserver_interface=127.23.194.1
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:39431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:03:20.971720 17492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b: Bootstrap starting.
I20260808 23:03:20.980581 17492 log.cc:824] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:20.982695 17492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:03:20.982952 17492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b: Bootstrap complete.
I20260808 23:03:20.984984 17492 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40997 } }
I20260808 23:03:20.985198 17492 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b9f5ac76f484b99a03e9d1cdeb8067b, State: Initialized, Role: FOLLOWER
I20260808 23:03:20.985334 17492 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40997 } }
I20260808 23:03:20.985422 17492 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:03:20.985471 17492 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:03:20.985520 17492 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 2 FOLLOWER]: Advancing to term 3
I20260808 23:03:20.986444 17492 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40997 } }
I20260808 23:03:20.986539 17492 leader_election.cc:302] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b9f5ac76f484b99a03e9d1cdeb8067b; no voters:
I20260808 23:03:20.986696 17492 leader_election.cc:288] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 23:03:20.986742 17498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 3 FOLLOWER]: Leader election won for term 3
I20260808 23:03:20.986864 17498 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [term 3 LEADER]: Becoming Leader. State: Replica: 3b9f5ac76f484b99a03e9d1cdeb8067b, State: Running, Role: LEADER
I20260808 23:03:20.986958 17492 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:03:20.986963 17498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40997 } }
I20260808 23:03:20.987442 17500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b9f5ac76f484b99a03e9d1cdeb8067b. Latest consensus state: current_term: 3 leader_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40997 } } }
I20260808 23:03:20.987569 17500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:20.987846 17499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9f5ac76f484b99a03e9d1cdeb8067b" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 40997 } } }
I20260808 23:03:20.988009 17499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:20.987936 17510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:03:20.988905 17510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:03:20.989005 17510 catalog_manager.cc:1259] Loaded cluster ID: f384cdec21564a56aa84ff73cd751acd
I20260808 23:03:20.989037 17510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:03:20.990208 17510 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:03:20.990373 17510 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b: Loaded TSK: 0
I20260808 23:03:20.990415 17510 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3b9f5ac76f484b99a03e9d1cdeb8067b: Loaded TSK: 1
F20260808 23:03:20.990644 17510 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f50f356f9ad google::LogMessage::Fail() at ??:0
@ 0x7f50f357288a google::LogMessage::SendToLog() at ??:0
@ 0x7f50f356f4de google::LogMessage::Flush() at ??:0
@ 0x7f50f3570a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f50f5cdc104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f50f5cd9d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f50fb4c91cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f50fb4c948e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f50fb4c9594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f50fb48fc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f50fb499ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f50f543d0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f50f5432954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f50f62966db start_thread at ??:0
@ 0x7f50f206d61f clone at ??:0
W20260808 23:03:21.074254 17496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:21.074463 17496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:21.074487 17496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:21.075662 17496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:21.075708 17496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:21.075736 17496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:03:21.077217 17496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:39431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:36185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=37173
--tserver_master_addrs=127.23.194.62:40997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:21.077487 17496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:21.077718 17496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:21.083999 17557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:21.083990 17558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:21.083990 17560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:21.084282 17496 server_base.cc:1034] running on GCE node
I20260808 23:03:21.084465 17496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:21.084704 17496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:21.085851 17496 hybrid_clock.cc:648] HybridClock initialized: now 1786230201085840 us; error 25 us; skew 500 ppm
Aug 08 23:03:21 dist-test-slave-f0tn krb5kdc[15839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230201, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:21.088425 17496 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 23:03:21.088878 17496 webserver.cc:460] Webserver started at http://127.23.194.1:37173/ using document root <none> and password file <none>
I20260808 23:03:21.089088 17496 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:21.089136 17496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:21.090533 17496 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:03:21.091248 17568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:21.091432 17496 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:21.091535 17496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "b48d7f8f597c49a1a998e304a3d36019"
format_stamp: "Formatted at 2026-08-08 23:02:39 on dist-test-slave-f0tn"
server_key: "6b5abe57553bcc71b67097698c041f1f"
server_key_iv: "bfc3396832466c9a8b086372a1b9d8c7"
server_key_version: "encryptionkey@0"
I20260808 23:03:21.091943 17496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:21.111747 17496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:21.114116 17496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:21.114283 17496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:21.114552 17496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:21.115058 17584 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 23:03:21.116504 17496 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 23:03:21.116564 17496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 23:03:21.116595 17496 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 23:03:21.117518 17496 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 23:03:21.117563 17496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260808 23:03:21.117619 17584 tablet_bootstrap.cc:492] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019: Bootstrap starting.
I20260808 23:03:21.119124 17584 log.cc:824] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:21.122040 17584 tablet_bootstrap.cc:492] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:03:21.122356 17584 tablet_bootstrap.cc:492] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019: Bootstrap complete.
I20260808 23:03:21.122510 17584 ts_tablet_manager.cc:1397] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 23:03:21.124182 17584 raft_consensus.cc:348] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } }
I20260808 23:03:21.124449 17584 raft_consensus.cc:729] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b48d7f8f597c49a1a998e304a3d36019, State: Initialized, Role: FOLLOWER
I20260808 23:03:21.124784 17496 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:36185
I20260808 23:03:21.124750 17584 consensus_queue.cc:260] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } }
I20260808 23:03:21.125036 17584 ts_tablet_manager.cc:1428] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 23:03:21.125182 17737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:36185 every 8 connection(s)
I20260808 23:03:21.125200 17496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:03:21.127225 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 17496
I20260808 23:03:21.127486 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:43831
--local_ip_for_outbound_sockets=127.23.194.2
--tserver_master_addrs=127.23.194.62:40997
--webserver_port=34769
--webserver_interface=127.23.194.2
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:39431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:21.201999 17744 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:03:21.125946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.1:58019)
0808 23:03:21.126101 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:21.126132 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:21.126268 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:21.201797 (+ 75529us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
W20260808 23:03:21.202239 17742 heartbeater.cc:644] Failed to heartbeat to 127.23.194.62:40997 (0 consecutive failures): Network error: Failed to ping master at 127.23.194.62:40997: Client connection negotiation failed: client connection to 127.23.194.62:40997: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 23:03:21.202598 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:21.202457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.1:52553)
0808 23:03:21.202510 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:21.202564 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 23:03:21.202944 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:21.202850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.1:44795)
0808 23:03:21.202896 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:21.202923 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 23:03:21.203051 17742 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 23:03:21.223251 17745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:21.223443 17745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:21.223465 17745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:21.224531 17745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:21.224565 17745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:21.224588 17745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:03:21.226030 17745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:39431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:43831
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=34769
--tserver_master_addrs=127.23.194.62:40997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:21.226220 17745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:21.226394 17745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:21.231168 17757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:21.231182 17756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:21.231201 17759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:21.231256 17745 server_base.cc:1034] running on GCE node
I20260808 23:03:21.231560 17745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:21.231777 17745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:21.232901 17745 hybrid_clock.cc:648] HybridClock initialized: now 1786230201232891 us; error 29 us; skew 500 ppm
Aug 08 23:03:21 dist-test-slave-f0tn krb5kdc[15839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230201, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:21.235312 17745 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 23:03:21.235718 17745 webserver.cc:460] Webserver started at http://127.23.194.2:34769/ using document root <none> and password file <none>
I20260808 23:03:21.235931 17745 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:21.235975 17745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:21.237197 17745 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:03:21.237778 17767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:21.237902 17745 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:21.237968 17745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "2092826ce0b442eda24e730644f09c57"
format_stamp: "Formatted at 2026-08-08 23:02:39 on dist-test-slave-f0tn"
server_key: "71853a32ec7f1eda357c7ade005c1d0d"
server_key_iv: "b39ae36bb899d3c23736c5c6929214b2"
server_key_version: "encryptionkey@0"
I20260808 23:03:21.238281 17745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:21.259052 17745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:21.261066 17745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:21.261193 17745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:21.261439 17745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:21.261756 17745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:21.261792 17745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:21.261814 17745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:21.261837 17745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:21.267438 17745 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:43831
I20260808 23:03:21.267503 17928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:43831 every 8 connection(s)
I20260808 23:03:21.267809 17745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:03:21.268802 17931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:21.268430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.2:57103)
0808 23:03:21.268577 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:21.268664 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
W20260808 23:03:21.269153 17929 heartbeater.cc:644] Failed to heartbeat to 127.23.194.62:40997 (0 consecutive failures): Network error: Failed to ping master at 127.23.194.62:40997: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
I20260808 23:03:21.269404 17931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:21.269312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.2:59431)
0808 23:03:21.269349 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:21.269388 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 23:03:21.269616 17931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:21.269557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.2:41337)
0808 23:03:21.269586 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:21.269604 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260808 23:03:21.269694 17929 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 23:03:21.273844 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 17745
I20260808 23:03:21.274081 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:36141
--local_ip_for_outbound_sockets=127.23.194.3
--tserver_master_addrs=127.23.194.62:40997
--webserver_port=42681
--webserver_interface=127.23.194.3
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:39431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:21.355232 17933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:21.355430 17933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:21.355453 17933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:21.356561 17933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:21.356602 17933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:21.356630 17933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:03:21.358109 17933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:39431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:36141
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=42681
--tserver_master_addrs=127.23.194.62:40997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:21.358340 17933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:21.358560 17933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:21.363603 17941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:21.363760 17933 server_base.cc:1034] running on GCE node
W20260808 23:03:21.363600 17940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:21.363613 17945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:21.364058 17933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:21.364293 17933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:21.365424 17933 hybrid_clock.cc:648] HybridClock initialized: now 1786230201365415 us; error 27 us; skew 500 ppm
Aug 08 23:03:21 dist-test-slave-f0tn krb5kdc[15839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230201, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:21.367835 17933 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 23:03:21.368225 17933 webserver.cc:460] Webserver started at http://127.23.194.3:42681/ using document root <none> and password file <none>
I20260808 23:03:21.368448 17933 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:21.368494 17933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:21.369807 17933 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 23:03:21.370384 17955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:21.370546 17933 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:03:21.370628 17933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "2087427f68454ae4920271588cd4af76"
format_stamp: "Formatted at 2026-08-08 23:02:39 on dist-test-slave-f0tn"
server_key: "94d061ae0b191e08dccc2b329577e80d"
server_key_iv: "a568eaa0be339e78d446faaa1f4b5095"
server_key_version: "encryptionkey@0"
I20260808 23:03:21.370936 17933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:21.392925 17933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:21.394783 17933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:21.394901 17933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:21.395095 17933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:21.395412 17933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:21.395447 17933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:21.395471 17933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:21.395494 17933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:21.401041 17933 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:36141
I20260808 23:03:21.401116 18116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:36141 every 8 connection(s)
I20260808 23:03:21.401454 17933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 23:03:21.402484 18121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:21.402097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.3:53591)
0808 23:03:21.402258 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:21.402333 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
W20260808 23:03:21.402860 18119 heartbeater.cc:644] Failed to heartbeat to 127.23.194.62:40997 (0 consecutive failures): Network error: Failed to ping master at 127.23.194.62:40997: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
I20260808 23:03:21.403119 18121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:21.403011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.3:38823)
0808 23:03:21.403067 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:21.403097 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 23:03:21.403363 18121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:21.403299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.3:42211)
0808 23:03:21.403329 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:21.403349 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260808 23:03:21.403447 18119 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 23:03:21.409451 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 17933
I20260808 23:03:22.203619 18123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:22.203391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.1:52139)
0808 23:03:22.203525 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:22.203557 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260808 23:03:22.270191 18124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:22.269974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.2:43861)
0808 23:03:22.270102 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:22.270138 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":57,"threads_started":1}
I20260808 23:03:22.403918 18125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:22.403723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.3:43515)
0808 23:03:22.403843 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:22.403874 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260808 23:03:22.697156 18126 raft_consensus.cc:482] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:03:22.697299 18126 raft_consensus.cc:504] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } }
I20260808 23:03:22.697726 18126 leader_election.cc:288] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2087427f68454ae4920271588cd4af76 (127.23.194.3:36141), 2092826ce0b442eda24e730644f09c57 (127.23.194.2:43831)
Aug 08 23:03:22 dist-test-slave-f0tn krb5kdc[15839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230201, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.3@KRBTEST.COM
Aug 08 23:03:22 dist-test-slave-f0tn krb5kdc[15839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230201, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.2@KRBTEST.COM
I20260808 23:03:22.708314 18129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:22.698056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:46585 (local address 127.23.194.3:36141)
0808 23:03:22.698222 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:22.698235 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:22.698268 (+ 33us) server_negotiation.cc:407] Connection header received
0808 23:03:22.698699 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:22.698701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:22.698743 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:22.698860 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:22.699930 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:22.700871 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:22.701928 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:22.702337 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:22.702444 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:22.704822 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:22.704826 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:22.704826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:22.704840 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:22.706754 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:22.707196 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:22.707199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:22.707200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:22.707288 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:22.707507 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:22.707508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:22.707508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:22.707730 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:22.707802 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:22.708074 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 23:03:22.708206 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260808 23:03:22.708315 18130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:22.698105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:37269 (local address 127.23.194.2:43831)
0808 23:03:22.698272 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:22.698283 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:22.698312 (+ 29us) server_negotiation.cc:407] Connection header received
0808 23:03:22.698746 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:22.698749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:22.698787 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:22.698920 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:22.700105 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:22.700950 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:22.701940 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:22.702338 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:22.702457 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:22.705150 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:22.705154 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:22.705155 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:22.705166 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:22.706853 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:22.707208 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:22.707211 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:22.707211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:22.707302 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:22.707526 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:22.707527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:22.707527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:22.707748 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:22.707824 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:22.708075 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 23:03:22.708194 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":67,"threads_started":1}
I20260808 23:03:22.708317 18128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:22.697893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:43831 (local address 127.23.194.1:37269)
0808 23:03:22.698119 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:22.698129 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:22.698174 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:22.699044 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:22.699045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:22.699438 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:22.699995 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:22.699999 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:22.701070 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:22.701071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:22.701853 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:22.701856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:22.702429 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:22.702430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:22.702544 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:22.703062 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:03:22.703070 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:22.705050 (+ 1980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:22.706918 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:22.706919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:22.706920 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:22.707134 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:22.707366 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:22.707367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:22.707367 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:22.707455 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:22.707815 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:22.707816 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:22.707924 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 23:03:22.708011 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 23:03:22.708200 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":34,"thread_start_us":49,"threads_started":1}
I20260808 23:03:22.708811 18123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:22.697874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:36141 (local address 127.23.194.1:46585)
0808 23:03:22.697915 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:22.697928 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:22.698029 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:22.698936 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:22.698939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:22.699436 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:22.699754 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:22.699763 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:22.700976 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:22.700977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:22.701853 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:22.701856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:22.702420 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:22.702421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:22.702543 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:22.703117 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:03:22.703122 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:22.704701 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:22.706837 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:22.706844 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:22.706846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:22.707113 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:22.707348 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:22.707349 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:22.707349 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:22.707451 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:22.707795 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:22.707801 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:22.707923 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 23:03:22.708007 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 23:03:22.708273 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":128}
W20260808 23:03:22.709208 17576 leader_election.cc:341] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2092826ce0b442eda24e730644f09c57 (127.23.194.2:43831): Not found: Tablet not found: 28bc4374866a4fbb859e2a5135746331
W20260808 23:03:22.709563 17571 leader_election.cc:341] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2087427f68454ae4920271588cd4af76 (127.23.194.3:36141): Not found: Tablet not found: 28bc4374866a4fbb859e2a5135746331
I20260808 23:03:22.709625 17571 leader_election.cc:302] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b48d7f8f597c49a1a998e304a3d36019; no voters: 2087427f68454ae4920271588cd4af76, 2092826ce0b442eda24e730644f09c57
I20260808 23:03:22.709749 18126 raft_consensus.cc:2726] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 23:03:23.204190 18123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:23.204047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.1:50493)
0808 23:03:23.204110 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:23.204147 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 23:03:23.270766 18131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:23.270576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.2:52405)
0808 23:03:23.270695 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:23.270729 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260808 23:03:23.404470 18133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:23.404295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.3:57433)
0808 23:03:23.404402 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:23.404434 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":58,"threads_started":1}
I20260808 23:03:24.204828 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:24.204624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.1:40687)
0808 23:03:24.204746 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:24.204781 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
I20260808 23:03:24.271387 18137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:24.271190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.2:33301)
0808 23:03:24.271315 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:24.271350 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260808 23:03:24.405049 18138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:24.404848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.3:46811)
0808 23:03:24.404976 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:24.405011 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
I20260808 23:03:24.599188 18139 raft_consensus.cc:482] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:03:24.599287 18139 raft_consensus.cc:504] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } }
I20260808 23:03:24.599424 18139 leader_election.cc:288] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2087427f68454ae4920271588cd4af76 (127.23.194.3:36141), 2092826ce0b442eda24e730644f09c57 (127.23.194.2:43831)
W20260808 23:03:24.599809 17571 leader_election.cc:341] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2087427f68454ae4920271588cd4af76 (127.23.194.3:36141): Not found: Tablet not found: 28bc4374866a4fbb859e2a5135746331
W20260808 23:03:24.599929 17576 leader_election.cc:341] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2092826ce0b442eda24e730644f09c57 (127.23.194.2:43831): Not found: Tablet not found: 28bc4374866a4fbb859e2a5135746331
I20260808 23:03:24.599982 17576 leader_election.cc:302] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b48d7f8f597c49a1a998e304a3d36019; no voters: 2087427f68454ae4920271588cd4af76, 2092826ce0b442eda24e730644f09c57
I20260808 23:03:24.600054 18139 raft_consensus.cc:2726] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 23:03:25.205484 18140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:25.205250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.1:53549)
0808 23:03:25.205368 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:25.205437 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
I20260808 23:03:25.271976 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:25.271774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.2:42985)
0808 23:03:25.271906 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:25.271939 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":67,"threads_started":1}
I20260808 23:03:25.405683 18142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:25.405474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.3:54319)
0808 23:03:25.405580 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:25.405618 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":56,"threads_started":1}
I20260808 23:03:26.206107 18143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:26.205901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.1:39659)
0808 23:03:26.206019 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:26.206057 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260808 23:03:26.272655 18144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:26.272437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.2:52019)
0808 23:03:26.272573 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:26.272611 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":61,"threads_started":1}
I20260808 23:03:26.406322 18145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:26.406121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.3:35279)
0808 23:03:26.406243 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:26.406279 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
I20260808 23:03:27.206723 18146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:27.206518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.1:48597)
0808 23:03:27.206641 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:27.206674 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260808 23:03:27.273343 18147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:27.273108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.2:52415)
0808 23:03:27.273262 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:27.273301 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260808 23:03:27.406939 18148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:27.406744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.3:55463)
0808 23:03:27.406862 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:27.406898 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260808 23:03:27.920244 18149 raft_consensus.cc:482] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:03:27.920346 18149 raft_consensus.cc:504] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } }
I20260808 23:03:27.920512 18149 leader_election.cc:288] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2087427f68454ae4920271588cd4af76 (127.23.194.3:36141), 2092826ce0b442eda24e730644f09c57 (127.23.194.2:43831)
W20260808 23:03:27.920904 17571 leader_election.cc:341] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2087427f68454ae4920271588cd4af76 (127.23.194.3:36141): Not found: Tablet not found: 28bc4374866a4fbb859e2a5135746331
W20260808 23:03:27.921006 17576 leader_election.cc:341] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2092826ce0b442eda24e730644f09c57 (127.23.194.2:43831): Not found: Tablet not found: 28bc4374866a4fbb859e2a5135746331
I20260808 23:03:27.921061 17576 leader_election.cc:302] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b48d7f8f597c49a1a998e304a3d36019; no voters: 2087427f68454ae4920271588cd4af76, 2092826ce0b442eda24e730644f09c57
I20260808 23:03:27.921147 18149 raft_consensus.cc:2726] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 23:03:28.207501 18150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:28.207264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.1:34753)
0808 23:03:28.207400 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:28.207452 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
I20260808 23:03:28.274016 18151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:28.273790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.2:43959)
0808 23:03:28.273922 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:28.273973 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260808 23:03:28.407591 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:28.407389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.3:34439)
0808 23:03:28.407505 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:28.407549 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260808 23:03:29.208190 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:29.207945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.1:43241)
0808 23:03:29.208097 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:29.208139 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
I20260808 23:03:29.274682 18155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:29.274459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.2:52819)
0808 23:03:29.274598 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:29.274637 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260808 23:03:29.408352 18157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:29.408078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.3:34161)
0808 23:03:29.408269 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:29.408308 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":105,"threads_started":1}
I20260808 23:03:30.208849 18159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:30.208628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.1:48885)
0808 23:03:30.208763 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:30.208801 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260808 23:03:30.275319 18161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:30.275086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.2:49117)
0808 23:03:30.275233 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:30.275273 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260808 23:03:30.408932 18162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:30.408734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.3:33449)
0808 23:03:30.408854 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:30.408890 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":57,"threads_started":1}
I20260808 23:03:31.209563 18163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:31.209323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.1:45103)
0808 23:03:31.209473 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:31.209508 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
I20260808 23:03:31.275930 18164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:31.275720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.2:59297)
0808 23:03:31.275842 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:31.275880 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
I20260808 23:03:31.409575 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:31.409347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.3:43597)
0808 23:03:31.409492 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:31.409528 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
I20260808 23:03:31.735054 18166 raft_consensus.cc:482] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:03:31.735144 18166 raft_consensus.cc:504] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2087427f68454ae4920271588cd4af76" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 36141 } } peers { permanent_uuid: "2092826ce0b442eda24e730644f09c57" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43831 } } peers { permanent_uuid: "b48d7f8f597c49a1a998e304a3d36019" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36185 } }
I20260808 23:03:31.735292 18166 leader_election.cc:288] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2087427f68454ae4920271588cd4af76 (127.23.194.3:36141), 2092826ce0b442eda24e730644f09c57 (127.23.194.2:43831)
W20260808 23:03:31.735641 17571 leader_election.cc:341] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2087427f68454ae4920271588cd4af76 (127.23.194.3:36141): Not found: Tablet not found: 28bc4374866a4fbb859e2a5135746331
W20260808 23:03:31.735736 17576 leader_election.cc:341] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2092826ce0b442eda24e730644f09c57 (127.23.194.2:43831): Not found: Tablet not found: 28bc4374866a4fbb859e2a5135746331
I20260808 23:03:31.735800 17576 leader_election.cc:302] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b48d7f8f597c49a1a998e304a3d36019; no voters: 2087427f68454ae4920271588cd4af76, 2092826ce0b442eda24e730644f09c57
I20260808 23:03:31.735880 18166 raft_consensus.cc:2726] T 28bc4374866a4fbb859e2a5135746331 P b48d7f8f597c49a1a998e304a3d36019 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 23:03:32.210263 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:32.210036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.1:58681)
0808 23:03:32.210164 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:32.210211 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260808 23:03:32.276504 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:32.276294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.2:58173)
0808 23:03:32.276428 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:32.276463 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
I20260808 23:03:32.410127 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:32.409945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.3:41367)
0808 23:03:32.410049 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:32.410084 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":58,"threads_started":1}
I20260808 23:03:33.210991 18173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:33.210759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.1:40579)
0808 23:03:33.210892 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:33.210940 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260808 23:03:33.277179 18174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:33.276937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.2:48533)
0808 23:03:33.277078 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:33.277128 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":61,"threads_started":1}
I20260808 23:03:33.410764 18176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:33.410549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.3:46053)
0808 23:03:33.410680 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:33.410724 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260808 23:03:34.211787 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:34.211521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.1:50845)
0808 23:03:34.211687 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:34.211739 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":68,"threads_started":1}
I20260808 23:03:34.277911 18179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:34.277654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.2:43765)
0808 23:03:34.277818 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:34.277866 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":80,"threads_started":1}
I20260808 23:03:34.411428 18181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:34.411215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.3:47215)
0808 23:03:34.411343 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:34.411387 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":59,"threads_started":1}
I20260808 23:03:35.212410 18182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:35.212209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.1:47197)
0808 23:03:35.212331 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:35.212362 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260808 23:03:35.278506 18185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:35.278322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.2:33381)
0808 23:03:35.278437 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:35.278466 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
I20260808 23:03:35.412036 18186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:35.411813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.3:56291)
0808 23:03:35.411960 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:35.411994 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":74,"threads_started":1}
I20260808 23:03:36.213109 18187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:36.212908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.1:58179)
0808 23:03:36.213025 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:36.213062 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":68,"threads_started":1}
I20260808 23:03:36.279232 18189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:36.279009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.2:53579)
0808 23:03:36.279145 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:36.279185 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260808 23:03:36.409780 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 17496
I20260808 23:03:36.418177 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:36.416017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:40997 (local address 127.23.194.3:52723)
0808 23:03:36.418069 (+ 2052us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:36.418126 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.62:40997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2026,"thread_start_us":55,"threads_started":1}
I20260808 23:03:36.422461 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 17745
I20260808 23:03:36.451957 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 17933
2026-08-08T23:03:36Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58411 ms)
[----------] 30 tests from SecurityITest (235249 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:03:36 dist-test-slave-f0tn krb5kdc[18196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:03:36 dist-test-slave-f0tn krb5kdc[18196](info): set up 2 sockets
Aug 08 23:03:36 dist-test-slave-f0tn krb5kdc[18196](info): commencing operation
krb5kdc: starting...
W20260808 23:03:37.596267 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:03:37 dist-test-slave-f0tn krb5kdc[18196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:03:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:03:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:03:37.709532 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:39363
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:35003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:39363
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:37.790983 18218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:37.791193 18218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:37.791227 18218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:37.792330 18218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:03:37.792376 18218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:37.792398 18218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:37.792418 18218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:03:37.792438 18218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:03:37.793973 18218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:35003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:39363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:39363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:37.794229 18218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:37.794436 18218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 23:03:37.799345 18218 server_base.cc:1034] running on GCE node
W20260808 23:03:37.799567 18228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:37.799374 18231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:37.799417 18229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:37.799914 18218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:37.800202 18218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:37.801354 18218 hybrid_clock.cc:648] HybridClock initialized: now 1786230217801335 us; error 35 us; skew 500 ppm
Aug 08 23:03:37 dist-test-slave-f0tn krb5kdc[18196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230217, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:37.803715 18218 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:03:37.804112 18218 webserver.cc:460] Webserver started at http://127.23.194.62:44279/ using document root <none> and password file <none>
I20260808 23:03:37.804308 18218 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:37.804347 18218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:37.804433 18218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:37.805421 18218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "8f8bb8484d144f4a9592c0c37d8f699c"
format_stamp: "Formatted at 2026-08-08 23:03:37 on dist-test-slave-f0tn"
server_key: "f701e92592a9648bce70de44837c2c18"
server_key_iv: "b3e4b81836ea43fe8f6baff77fe28f0c"
server_key_version: "encryptionkey@0"
I20260808 23:03:37.805727 18218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "8f8bb8484d144f4a9592c0c37d8f699c"
format_stamp: "Formatted at 2026-08-08 23:03:37 on dist-test-slave-f0tn"
server_key: "f701e92592a9648bce70de44837c2c18"
server_key_iv: "b3e4b81836ea43fe8f6baff77fe28f0c"
server_key_version: "encryptionkey@0"
I20260808 23:03:37.806870 18218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:03:37.807497 18242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:37.807665 18218 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:37.807749 18218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "8f8bb8484d144f4a9592c0c37d8f699c"
format_stamp: "Formatted at 2026-08-08 23:03:37 on dist-test-slave-f0tn"
server_key: "f701e92592a9648bce70de44837c2c18"
server_key_iv: "b3e4b81836ea43fe8f6baff77fe28f0c"
server_key_version: "encryptionkey@0"
I20260808 23:03:37.807802 18218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:37.827343 18218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:37.829083 18218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:37.829185 18218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:37.832938 18218 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:39363
I20260808 23:03:37.833026 18314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:39363 every 8 connection(s)
I20260808 23:03:37.833297 18218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:03:37.833557 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 18218
I20260808 23:03:37.833669 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:03:37.833853 24328 external_mini_cluster.cc:1428] Setting key dd2bc30fb8834ea1e45af46ea9560632
I20260808 23:03:37.834263 18316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:37.837292 18316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c: Bootstrap starting.
I20260808 23:03:37.837975 18316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:37.838255 18316 log.cc:824] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:37.839042 18316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c: No bootstrap required, opened a new log
Aug 08 23:03:37 dist-test-slave-f0tn krb5kdc[18196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:03:37.841130 18316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f8bb8484d144f4a9592c0c37d8f699c" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39363 } }
I20260808 23:03:37.841296 18316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:37.841346 18316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f8bb8484d144f4a9592c0c37d8f699c, State: Initialized, Role: FOLLOWER
I20260808 23:03:37.841498 18316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f8bb8484d144f4a9592c0c37d8f699c" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39363 } }
I20260808 23:03:37.841589 18316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:03:37.841622 18316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:03:37.841676 18316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:37.842206 18316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f8bb8484d144f4a9592c0c37d8f699c" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39363 } }
I20260808 23:03:37.842294 18316 leader_election.cc:302] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f8bb8484d144f4a9592c0c37d8f699c; no voters:
I20260808 23:03:37.842571 18316 leader_election.cc:288] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:03:37.842615 18323 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:37.842727 18323 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c [term 1 LEADER]: Becoming Leader. State: Replica: 8f8bb8484d144f4a9592c0c37d8f699c, State: Running, Role: LEADER
I20260808 23:03:37.842840 18316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:03:37.842810 18323 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f8bb8484d144f4a9592c0c37d8f699c" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39363 } }
I20260808 23:03:37.843218 18325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f8bb8484d144f4a9592c0c37d8f699c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f8bb8484d144f4a9592c0c37d8f699c" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39363 } } }
I20260808 23:03:37.843300 18325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:37.843494 18330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:03:37.844077 18330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:03:37.843179 18323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f8bb8484d144f4a9592c0c37d8f699c. Latest consensus state: current_term: 1 leader_uuid: "8f8bb8484d144f4a9592c0c37d8f699c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f8bb8484d144f4a9592c0c37d8f699c" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39363 } } }
I20260808 23:03:37.844511 18323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:37.844571 18320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:37.834317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54126 (local address 127.23.194.62:39363)
0808 23:03:37.834469 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:37.834478 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:37.834506 (+ 28us) server_negotiation.cc:407] Connection header received
0808 23:03:37.834930 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:37.834936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:37.834979 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:37.835084 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:37.835854 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:37.836703 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:37.837745 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:37.838161 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:37.838240 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:37.840771 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:37.840775 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:37.840775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:37.840785 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:37.842484 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:37.842867 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:37.842870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:37.842871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:37.842974 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:37.843181 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:37.843182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:37.843182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:37.843522 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:37.843675 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:37.844203 (+ 528us) server_negotiation.cc:299] Negotiation successful
0808 23:03:37.844349 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
I20260808 23:03:37.845675 18330 catalog_manager.cc:1347] Generated new cluster ID: 3b049fe255404a61b4561c11ccbf4e69
I20260808 23:03:37.845734 18330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:03:37.856091 18330 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:03:37.856755 18330 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:03:37.863035 18330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c: Generated new TSK 0
I20260808 23:03:37.863209 18330 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
I20260808 23:03:37.892004 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:39363
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:35003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:37.992238 18349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:37.992429 18349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:37.992450 18349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:37.993592 18349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:37.993630 18349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:37.993655 18349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:03:37.995090 18349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:35003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:39363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:37.995302 18349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:37.995512 18349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:38.000315 18360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:38.000358 18359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:38.000494 18363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:38.000509 18349 server_base.cc:1034] running on GCE node
I20260808 23:03:38.000728 18349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:38.000946 18349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:38.002074 18349 hybrid_clock.cc:648] HybridClock initialized: now 1786230218002062 us; error 33 us; skew 500 ppm
Aug 08 23:03:37 dist-test-slave-f0tn krb5kdc[18196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230217, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:38.004446 18349 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 23:03:38.004848 18349 webserver.cc:460] Webserver started at http://127.23.194.1:44271/ using document root <none> and password file <none>
I20260808 23:03:38.005065 18349 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:38.005108 18349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:38.005237 18349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:38.006291 18349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "9f252c5974bb47c4bb8b81906f945efc"
format_stamp: "Formatted at 2026-08-08 23:03:38 on dist-test-slave-f0tn"
server_key: "7c95920bd8dac944e171b09564d4e69f"
server_key_iv: "5f7f3dfa6dc80a684e2f2d163a48990f"
server_key_version: "encryptionkey@0"
I20260808 23:03:38.006609 18349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "9f252c5974bb47c4bb8b81906f945efc"
format_stamp: "Formatted at 2026-08-08 23:03:38 on dist-test-slave-f0tn"
server_key: "7c95920bd8dac944e171b09564d4e69f"
server_key_iv: "5f7f3dfa6dc80a684e2f2d163a48990f"
server_key_version: "encryptionkey@0"
I20260808 23:03:38.007833 18349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:03:38.008487 18373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:38.008664 18349 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:38.008744 18349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "9f252c5974bb47c4bb8b81906f945efc"
format_stamp: "Formatted at 2026-08-08 23:03:38 on dist-test-slave-f0tn"
server_key: "7c95920bd8dac944e171b09564d4e69f"
server_key_iv: "5f7f3dfa6dc80a684e2f2d163a48990f"
server_key_version: "encryptionkey@0"
I20260808 23:03:38.008801 18349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:38.024931 18349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:38.027132 18349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:38.027251 18349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:38.027500 18349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:38.027767 18349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:38.027801 18349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:38.027825 18349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:38.027844 18349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:38.033809 18349 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:45471
I20260808 23:03:38.033849 18535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:45471 every 8 connection(s)
I20260808 23:03:38.034157 18349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:03:38.037261 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 18349
I20260808 23:03:38.037354 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:03:38.037559 24328 external_mini_cluster.cc:1428] Setting key 56bfb821f2f0e36ecb5b9abf4efeccb5
Aug 08 23:03:38 dist-test-slave-f0tn krb5kdc[18196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230217, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:03:38.043324 18320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.034841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:47007 (local address 127.23.194.62:39363)
0808 23:03:38.034897 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:38.034899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:38.035076 (+ 177us) server_negotiation.cc:407] Connection header received
0808 23:03:38.035543 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:38.035544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:38.035555 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:38.035601 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:38.036463 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.036956 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.038018 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.038295 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.038361 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.040578 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:38.040580 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:38.040580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:38.040589 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:38.041971 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:38.042363 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:38.042364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:38.042365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:38.042424 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:38.042662 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:38.042663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:38.042663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:38.042778 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:38.042835 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:38.043131 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 23:03:38.043230 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 23:03:38.043442 18539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.034783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:39363 (local address 127.23.194.1:47007)
0808 23:03:38.034954 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:38.034995 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:38.035447 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:38.035672 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:38.035675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:38.036212 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:38.036394 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:38.036399 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.037046 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.037047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:38.037916 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:38.037919 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.038397 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.038399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:38.038546 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.038998 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:03:38.039008 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:38.040465 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:38.042051 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:38.042058 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:38.042060 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:38.042294 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:38.042479 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:38.042480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:38.042480 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:38.042578 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:38.042836 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:38.042844 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:38.042990 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 23:03:38.043083 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 23:03:38.043240 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260808 23:03:38.044122 18536 heartbeater.cc:347] Connected to a master server at 127.23.194.62:39363
I20260808 23:03:38.044206 18536 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:38.044368 18536 heartbeater.cc:511] Master 127.23.194.62:39363 requested a full tablet report, sending...
I20260808 23:03:38.044878 18264 ts_manager.cc:194] Registered new tserver with Master: 9f252c5974bb47c4bb8b81906f945efc (127.23.194.1:45471)
I20260808 23:03:38.045533 18264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:47007
WARNING: no policy specified for kudu/127.23.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
I20260808 23:03:38.061075 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:39363
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:35003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:38.142366 18543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:38.142566 18543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:38.142589 18543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:38.143807 18543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:38.143852 18543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:38.143888 18543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:03:38.145403 18543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:35003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:39363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:38.145635 18543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:38.145833 18543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:38.150617 18552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:38.150604 18554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:38.150738 18543 server_base.cc:1034] running on GCE node
W20260808 23:03:38.150789 18551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:38.151050 18543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:38.151285 18543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:38.152424 18543 hybrid_clock.cc:648] HybridClock initialized: now 1786230218152413 us; error 29 us; skew 500 ppm
Aug 08 23:03:38 dist-test-slave-f0tn krb5kdc[18196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230218, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:38.154835 18543 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 23:03:38.155270 18543 webserver.cc:460] Webserver started at http://127.23.194.2:46033/ using document root <none> and password file <none>
I20260808 23:03:38.155493 18543 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:38.155539 18543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:38.155671 18543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:38.156680 18543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "9c68ed834dcd4e01a33561afe8b6bc11"
format_stamp: "Formatted at 2026-08-08 23:03:38 on dist-test-slave-f0tn"
server_key: "27f1f9b809968466dd7ce0b78089df7f"
server_key_iv: "ebf80b00d0c216e6e5530f66f707b212"
server_key_version: "encryptionkey@0"
I20260808 23:03:38.156991 18543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "9c68ed834dcd4e01a33561afe8b6bc11"
format_stamp: "Formatted at 2026-08-08 23:03:38 on dist-test-slave-f0tn"
server_key: "27f1f9b809968466dd7ce0b78089df7f"
server_key_iv: "ebf80b00d0c216e6e5530f66f707b212"
server_key_version: "encryptionkey@0"
I20260808 23:03:38.158285 18543 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 23:03:38.158929 18564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:38.159094 18543 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:03:38.159174 18543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "9c68ed834dcd4e01a33561afe8b6bc11"
format_stamp: "Formatted at 2026-08-08 23:03:38 on dist-test-slave-f0tn"
server_key: "27f1f9b809968466dd7ce0b78089df7f"
server_key_iv: "ebf80b00d0c216e6e5530f66f707b212"
server_key_version: "encryptionkey@0"
I20260808 23:03:38.159227 18543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:38.170652 18543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:38.172291 18543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:38.172400 18543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:38.172566 18543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:38.172813 18543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:38.172844 18543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:38.172863 18543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:38.172883 18543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:38.178233 18543 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:34973
I20260808 23:03:38.178308 18724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:34973 every 8 connection(s)
I20260808 23:03:38.178568 18543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
Aug 08 23:03:38 dist-test-slave-f0tn krb5kdc[18196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230218, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:03:38.186111 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 18543
I20260808 23:03:38.186237 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:03:38.186440 24328 external_mini_cluster.cc:1428] Setting key 0ddbd39223bcae4cf756ca9daaa3f555
I20260808 23:03:38.187125 18320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.179247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:55983 (local address 127.23.194.62:39363)
0808 23:03:38.179319 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:38.179321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:38.179476 (+ 155us) server_negotiation.cc:407] Connection header received
0808 23:03:38.179961 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:38.179962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:38.179972 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:38.180007 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:38.180885 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.181371 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.182333 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.182523 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.182567 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.184510 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:38.184512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:38.184512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:38.184519 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:38.185785 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:38.186130 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:38.186132 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:38.186132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:38.186210 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:38.186463 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:38.186464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:38.186465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:38.186601 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:38.186662 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:38.186964 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 23:03:38.187050 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 23:03:38.187335 18727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.179208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:39363 (local address 127.23.194.2:55983)
0808 23:03:38.179365 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:38.179400 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:38.179858 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:38.180072 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:38.180075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:38.180624 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:38.180821 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:38.180826 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.181462 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.181463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:38.182269 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:38.182271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.182575 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.182575 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:38.182675 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.183120 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:03:38.183128 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:38.184420 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:38.185852 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:38.185858 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:38.185859 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:38.186063 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:38.186293 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:38.186294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:38.186294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:38.186404 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:38.186679 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:38.186686 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:38.186838 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 23:03:38.186945 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 23:03:38.187107 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
I20260808 23:03:38.187739 18725 heartbeater.cc:347] Connected to a master server at 127.23.194.62:39363
I20260808 23:03:38.187824 18725 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:38.187986 18725 heartbeater.cc:511] Master 127.23.194.62:39363 requested a full tablet report, sending...
I20260808 23:03:38.188331 18264 ts_manager.cc:194] Registered new tserver with Master: 9c68ed834dcd4e01a33561afe8b6bc11 (127.23.194.2:34973)
I20260808 23:03:38.188685 18264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:55983
WARNING: no policy specified for kudu/127.23.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
I20260808 23:03:38.209913 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:39363
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:35003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:38.292446 18730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:38.292665 18730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:38.292695 18730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:38.293869 18730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:38.293910 18730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:38.293944 18730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:03:38.295370 18730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:35003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:39363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:38.295612 18730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:38.295814 18730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:38.300730 18740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:38.300750 18743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:38.300738 18737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:38.300951 18730 server_base.cc:1034] running on GCE node
I20260808 23:03:38.301108 18730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:38.301316 18730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:38.302455 18730 hybrid_clock.cc:648] HybridClock initialized: now 1786230218302442 us; error 31 us; skew 500 ppm
Aug 08 23:03:38 dist-test-slave-f0tn krb5kdc[18196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230218, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:38.304797 18730 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 23:03:38.305213 18730 webserver.cc:460] Webserver started at http://127.23.194.3:35441/ using document root <none> and password file <none>
I20260808 23:03:38.305423 18730 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:38.305467 18730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:38.305562 18730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:38.306527 18730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "565fc27a820b490e871e5b0d5eda1b8b"
format_stamp: "Formatted at 2026-08-08 23:03:38 on dist-test-slave-f0tn"
server_key: "e7c50a83aff0514f74f5674867f40f6f"
server_key_iv: "aeb3d7a0b940eeb5be5ac5dc2806a13e"
server_key_version: "encryptionkey@0"
I20260808 23:03:38.306819 18730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "565fc27a820b490e871e5b0d5eda1b8b"
format_stamp: "Formatted at 2026-08-08 23:03:38 on dist-test-slave-f0tn"
server_key: "e7c50a83aff0514f74f5674867f40f6f"
server_key_iv: "aeb3d7a0b940eeb5be5ac5dc2806a13e"
server_key_version: "encryptionkey@0"
I20260808 23:03:38.307969 18730 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 23:03:38.308579 18752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:38.308718 18730 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:03:38.308797 18730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "565fc27a820b490e871e5b0d5eda1b8b"
format_stamp: "Formatted at 2026-08-08 23:03:38 on dist-test-slave-f0tn"
server_key: "e7c50a83aff0514f74f5674867f40f6f"
server_key_iv: "aeb3d7a0b940eeb5be5ac5dc2806a13e"
server_key_version: "encryptionkey@0"
I20260808 23:03:38.308861 18730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:38.318357 18730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:38.320077 18730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:38.320202 18730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:38.320421 18730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:38.320727 18730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:38.320762 18730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:38.320786 18730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:38.320808 18730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:38.326551 18730 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:38191
I20260808 23:03:38.326608 18920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:38191 every 8 connection(s)
I20260808 23:03:38.326915 18730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
Aug 08 23:03:38 dist-test-slave-f0tn krb5kdc[18196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230218, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:03:38.335515 18320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.327618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:51117 (local address 127.23.194.62:39363)
0808 23:03:38.327703 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:38.327705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:38.327850 (+ 145us) server_negotiation.cc:407] Connection header received
0808 23:03:38.328325 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:38.328326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:38.328336 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:38.328388 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:38.329316 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.329921 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.330885 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.331085 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.331143 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.333148 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:38.333149 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:38.333150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:38.333158 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:38.334269 (+ 1111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:38.334624 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:38.334626 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:38.334626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:38.334695 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:38.334913 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:38.334914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:38.334914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:38.335021 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:38.335077 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:38.335340 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 23:03:38.335423 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 23:03:38.335680 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.327586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:39363 (local address 127.23.194.3:51117)
0808 23:03:38.327734 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:38.327772 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:38.328252 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:38.328444 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:38.328447 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:38.329015 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:38.329231 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:38.329237 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.330005 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.330006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:38.330817 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:38.330820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.331187 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.331188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:38.331322 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.331745 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:03:38.331753 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:38.333057 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:38.334338 (+ 1281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:38.334345 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:38.334347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:38.334558 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:38.334752 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:38.334752 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:38.334753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:38.334858 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:38.335076 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:38.335083 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:38.335222 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 23:03:38.335310 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 23:03:38.335467 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
I20260808 23:03:38.335903 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 18730
I20260808 23:03:38.336022 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:03:38.336163 18921 heartbeater.cc:347] Connected to a master server at 127.23.194.62:39363
I20260808 23:03:38.336213 24328 external_mini_cluster.cc:1428] Setting key cdef20a985da7b655edf4d624dde2545
I20260808 23:03:38.336246 18921 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:38.336404 18921 heartbeater.cc:511] Master 127.23.194.62:39363 requested a full tablet report, sending...
I20260808 23:03:38.336745 18264 ts_manager.cc:194] Registered new tserver with Master: 565fc27a820b490e871e5b0d5eda1b8b (127.23.194.3:38191)
I20260808 23:03:38.337191 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:03:38.337242 18264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:51117
I20260808 23:03:38.343788 18320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.338544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54140 (local address 127.23.194.62:39363)
0808 23:03:38.338581 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:38.338582 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:38.338678 (+ 96us) server_negotiation.cc:407] Connection header received
0808 23:03:38.338725 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:38.338725 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:38.338734 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:38.338777 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:38.339329 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.339775 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.340415 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.340585 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.340626 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.341744 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:38.341745 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:38.341745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:38.341751 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:38.342833 (+ 1082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:38.343120 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:38.343120 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:38.343120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:38.343178 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:38.343372 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:38.343373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:38.343373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:38.343504 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:38.343585 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:38.343665 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 23:03:38.343725 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 23:03:38.345355 18264 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:03:38.346995 18264 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:03:38.353166 18941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.349123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:34973 (local address 127.0.0.1:48564)
0808 23:03:38.349518 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:38.349528 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:38.349582 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:38.349771 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:38.349772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:38.349774 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:38.349820 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:38.349821 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.351100 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.351102 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:38.352198 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:38.352199 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.352862 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.352863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:38.352940 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.352955 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 23:03:38.353020 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 23:03:38.353078 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":37,"threads_started":1}
I20260808 23:03:38.353165 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.349309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48564 (local address 127.23.194.2:34973)
0808 23:03:38.349516 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:38.349520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:38.349574 (+ 54us) server_negotiation.cc:407] Connection header received
0808 23:03:38.349643 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:38.349645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:38.349660 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:38.349774 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:38.349887 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.350996 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.352270 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.352782 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.352847 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.352913 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:38.353043 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 23:03:38.353088 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":126,"threads_started":1}
I20260808 23:03:38.353298 18936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.348930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:45471 (local address 127.0.0.1:49284)
0808 23:03:38.349093 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:38.349132 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:38.349211 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:38.349488 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:38.349491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:38.349494 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:38.349567 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:38.349569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.351119 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.351120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:38.352039 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:38.352042 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.353030 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.353031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:38.353111 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.353115 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:03:38.353173 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 23:03:38.353219 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":41,"thread_start_us":63,"threads_started":1}
I20260808 23:03:38.353407 18939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.349014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49708 (local address 127.23.194.3:38191)
0808 23:03:38.349734 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:38.349738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:38.349747 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:03:38.349787 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:38.349789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:38.349805 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:38.349879 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:38.350078 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.350761 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.352176 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.352871 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.352937 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.352992 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:38.353267 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 23:03:38.353313 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":110,"threads_started":1}
I20260808 23:03:38.353534 18937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.348930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:38191 (local address 127.0.0.1:49708)
0808 23:03:38.349306 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:38.349313 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:38.349349 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:38.349962 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:38.349962 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:38.349965 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:38.350003 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:38.350004 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.350855 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.350856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:38.352104 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:38.352107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.353031 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.353032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:38.353127 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.353131 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:03:38.353173 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 23:03:38.353219 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":57,"spinlock_wait_cycles":896,"thread_start_us":56,"threads_started":1}
I20260808 23:03:38.353535 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.349111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49284 (local address 127.23.194.1:45471)
0808 23:03:38.349296 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:38.349301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:38.349312 (+ 11us) server_negotiation.cc:407] Connection header received
0808 23:03:38.349347 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:38.349349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:38.349361 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:38.349483 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:38.349641 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.351012 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.352117 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.352868 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.352938 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.352992 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:38.353411 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 23:03:38.353456 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":90,"threads_started":1}
I20260808 23:03:38.354175 18446 tablet_service.cc:1467] Processing CreateTablet for tablet bf6d12379e5842caacdf98326a388265 (DEFAULT_TABLE table=test-table [id=da8a4ec24b7e409493f17bbf741ee4e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:38.354271 18821 tablet_service.cc:1467] Processing CreateTablet for tablet bf6d12379e5842caacdf98326a388265 (DEFAULT_TABLE table=test-table [id=da8a4ec24b7e409493f17bbf741ee4e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:38.354526 18446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6d12379e5842caacdf98326a388265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:38.354580 18821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6d12379e5842caacdf98326a388265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:38.357473 18945 tablet_bootstrap.cc:492] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b: Bootstrap starting.
I20260808 23:03:38.357653 18947 tablet_bootstrap.cc:492] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc: Bootstrap starting.
I20260808 23:03:38.358177 18945 tablet_bootstrap.cc:654] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:38.358299 18947 tablet_bootstrap.cc:654] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:38.358696 18945 log.cc:824] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:38.359244 18632 tablet_service.cc:1467] Processing CreateTablet for tablet bf6d12379e5842caacdf98326a388265 (DEFAULT_TABLE table=test-table [id=da8a4ec24b7e409493f17bbf741ee4e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:38.359539 18632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6d12379e5842caacdf98326a388265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:38.359665 18945 tablet_bootstrap.cc:492] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b: No bootstrap required, opened a new log
I20260808 23:03:38.359735 18945 ts_tablet_manager.cc:1397] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 23:03:38.361217 18945 raft_consensus.cc:348] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f252c5974bb47c4bb8b81906f945efc" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 45471 } } peers { permanent_uuid: "565fc27a820b490e871e5b0d5eda1b8b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38191 } } peers { permanent_uuid: "9c68ed834dcd4e01a33561afe8b6bc11" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 34973 } }
I20260808 23:03:38.361402 18945 raft_consensus.cc:374] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:38.358850 18947 log.cc:824] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:38.361444 18945 raft_consensus.cc:729] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 565fc27a820b490e871e5b0d5eda1b8b, State: Initialized, Role: FOLLOWER
I20260808 23:03:38.361555 18945 consensus_queue.cc:260] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f252c5974bb47c4bb8b81906f945efc" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 45471 } } peers { permanent_uuid: "565fc27a820b490e871e5b0d5eda1b8b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38191 } } peers { permanent_uuid: "9c68ed834dcd4e01a33561afe8b6bc11" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 34973 } }
I20260808 23:03:38.361838 18945 ts_tablet_manager.cc:1428] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 23:03:38.361967 18921 heartbeater.cc:503] Master 127.23.194.62:39363 was elected leader, sending a full tablet report...
I20260808 23:03:38.362375 18947 tablet_bootstrap.cc:492] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc: No bootstrap required, opened a new log
I20260808 23:03:38.362443 18947 ts_tablet_manager.cc:1397] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 23:03:38.362685 18951 tablet_bootstrap.cc:492] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11: Bootstrap starting.
I20260808 23:03:38.363372 18951 tablet_bootstrap.cc:654] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:38.363931 18951 log.cc:824] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:38.364455 18947 raft_consensus.cc:348] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f252c5974bb47c4bb8b81906f945efc" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 45471 } } peers { permanent_uuid: "565fc27a820b490e871e5b0d5eda1b8b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38191 } } peers { permanent_uuid: "9c68ed834dcd4e01a33561afe8b6bc11" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 34973 } }
I20260808 23:03:38.364639 18951 tablet_bootstrap.cc:492] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11: No bootstrap required, opened a new log
I20260808 23:03:38.364647 18947 raft_consensus.cc:374] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:38.364686 18947 raft_consensus.cc:729] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f252c5974bb47c4bb8b81906f945efc, State: Initialized, Role: FOLLOWER
I20260808 23:03:38.364705 18951 ts_tablet_manager.cc:1397] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 23:03:38.364806 18947 consensus_queue.cc:260] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f252c5974bb47c4bb8b81906f945efc" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 45471 } } peers { permanent_uuid: "565fc27a820b490e871e5b0d5eda1b8b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38191 } } peers { permanent_uuid: "9c68ed834dcd4e01a33561afe8b6bc11" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 34973 } }
I20260808 23:03:38.365077 18947 ts_tablet_manager.cc:1428] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 23:03:38.365095 18536 heartbeater.cc:503] Master 127.23.194.62:39363 was elected leader, sending a full tablet report...
I20260808 23:03:38.366180 18951 raft_consensus.cc:348] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f252c5974bb47c4bb8b81906f945efc" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 45471 } } peers { permanent_uuid: "565fc27a820b490e871e5b0d5eda1b8b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38191 } } peers { permanent_uuid: "9c68ed834dcd4e01a33561afe8b6bc11" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 34973 } }
I20260808 23:03:38.366304 18951 raft_consensus.cc:374] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:38.366323 18951 raft_consensus.cc:729] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c68ed834dcd4e01a33561afe8b6bc11, State: Initialized, Role: FOLLOWER
I20260808 23:03:38.366400 18951 consensus_queue.cc:260] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f252c5974bb47c4bb8b81906f945efc" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 45471 } } peers { permanent_uuid: "565fc27a820b490e871e5b0d5eda1b8b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38191 } } peers { permanent_uuid: "9c68ed834dcd4e01a33561afe8b6bc11" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 34973 } }
I20260808 23:03:38.366593 18951 ts_tablet_manager.cc:1428] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 23:03:38.366647 18725 heartbeater.cc:503] Master 127.23.194.62:39363 was elected leader, sending a full tablet report...
W20260808 23:03:38.429188 18726 tablet.cc:2369] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:03:38.441455 18955 raft_consensus.cc:482] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:03:38.441591 18955 raft_consensus.cc:504] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f252c5974bb47c4bb8b81906f945efc" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 45471 } } peers { permanent_uuid: "565fc27a820b490e871e5b0d5eda1b8b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38191 } } peers { permanent_uuid: "9c68ed834dcd4e01a33561afe8b6bc11" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 34973 } }
I20260808 23:03:38.441919 18955 leader_election.cc:288] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 565fc27a820b490e871e5b0d5eda1b8b (127.23.194.3:38191), 9c68ed834dcd4e01a33561afe8b6bc11 (127.23.194.2:34973)
I20260808 23:03:38.445791 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.442216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:37519 (local address 127.23.194.2:34973)
0808 23:03:38.442268 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:38.442270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:38.442403 (+ 133us) server_negotiation.cc:407] Connection header received
0808 23:03:38.442481 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:38.442482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:38.442490 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:38.442539 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:38.442660 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.443358 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.444821 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.445427 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.445501 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.445509 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:38.445679 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 23:03:38.445719 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 23:03:38.445791 18939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.442209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:44961 (local address 127.23.194.3:38191)
0808 23:03:38.442258 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:38.442260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:38.442269 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:03:38.442297 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:38.442298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:38.442307 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:38.442380 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:38.442541 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.443323 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.444821 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.445431 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.445500 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.445509 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:38.445679 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 23:03:38.445719 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 23:03:38.445823 18539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.442087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:38191 (local address 127.23.194.1:44961)
0808 23:03:38.442149 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:38.442163 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:38.442218 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:38.442389 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:38.442390 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:38.442392 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:38.442470 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:38.442473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.443403 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.443404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:38.444727 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:38.444731 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.445512 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.445513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:38.445612 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.445614 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 23:03:38.445667 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 23:03:38.445713 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":45}
I20260808 23:03:38.445828 18958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.442087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:34973 (local address 127.23.194.1:37519)
0808 23:03:38.442332 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:38.442340 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:38.442425 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:38.442553 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:38.442554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:38.442555 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:38.442609 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:38.442610 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.443418 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.443418 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:38.444727 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:38.444731 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.445513 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.445514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:38.445597 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.445602 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:03:38.445669 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 23:03:38.445712 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":87,"thread_start_us":123,"threads_started":1}
I20260808 23:03:38.446197 18852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6d12379e5842caacdf98326a388265" candidate_uuid: "9f252c5974bb47c4bb8b81906f945efc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565fc27a820b490e871e5b0d5eda1b8b" is_pre_election: true
I20260808 23:03:38.446197 18659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6d12379e5842caacdf98326a388265" candidate_uuid: "9f252c5974bb47c4bb8b81906f945efc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c68ed834dcd4e01a33561afe8b6bc11" is_pre_election: true
I20260808 23:03:38.446337 18659 raft_consensus.cc:2454] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f252c5974bb47c4bb8b81906f945efc in term 0.
I20260808 23:03:38.446342 18852 raft_consensus.cc:2454] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f252c5974bb47c4bb8b81906f945efc in term 0.
I20260808 23:03:38.446557 18379 leader_election.cc:302] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 565fc27a820b490e871e5b0d5eda1b8b, 9f252c5974bb47c4bb8b81906f945efc; no voters:
I20260808 23:03:38.446704 18955 raft_consensus.cc:2781] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:03:38.446763 18955 raft_consensus.cc:482] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:03:38.446785 18955 raft_consensus.cc:3037] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:38.447485 18955 raft_consensus.cc:504] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f252c5974bb47c4bb8b81906f945efc" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 45471 } } peers { permanent_uuid: "565fc27a820b490e871e5b0d5eda1b8b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38191 } } peers { permanent_uuid: "9c68ed834dcd4e01a33561afe8b6bc11" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 34973 } }
I20260808 23:03:38.447608 18955 leader_election.cc:288] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc [CANDIDATE]: Term 1 election: Requested vote from peers 565fc27a820b490e871e5b0d5eda1b8b (127.23.194.3:38191), 9c68ed834dcd4e01a33561afe8b6bc11 (127.23.194.2:34973)
I20260808 23:03:38.447753 18659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6d12379e5842caacdf98326a388265" candidate_uuid: "9f252c5974bb47c4bb8b81906f945efc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c68ed834dcd4e01a33561afe8b6bc11"
I20260808 23:03:38.447775 18852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6d12379e5842caacdf98326a388265" candidate_uuid: "9f252c5974bb47c4bb8b81906f945efc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565fc27a820b490e871e5b0d5eda1b8b"
I20260808 23:03:38.447841 18852 raft_consensus.cc:3037] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:38.447842 18659 raft_consensus.cc:3037] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:38.448729 18852 raft_consensus.cc:2454] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f252c5974bb47c4bb8b81906f945efc in term 1.
I20260808 23:03:38.448729 18659 raft_consensus.cc:2454] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f252c5974bb47c4bb8b81906f945efc in term 1.
I20260808 23:03:38.448896 18379 leader_election.cc:302] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 565fc27a820b490e871e5b0d5eda1b8b, 9f252c5974bb47c4bb8b81906f945efc; no voters:
I20260808 23:03:38.448993 18955 raft_consensus.cc:2781] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:38.449113 18955 raft_consensus.cc:686] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc [term 1 LEADER]: Becoming Leader. State: Replica: 9f252c5974bb47c4bb8b81906f945efc, State: Running, Role: LEADER
I20260808 23:03:38.449196 18955 consensus_queue.cc:237] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f252c5974bb47c4bb8b81906f945efc" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 45471 } } peers { permanent_uuid: "565fc27a820b490e871e5b0d5eda1b8b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38191 } } peers { permanent_uuid: "9c68ed834dcd4e01a33561afe8b6bc11" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 34973 } }
I20260808 23:03:38.450079 18264 catalog_manager.cc:5495] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f252c5974bb47c4bb8b81906f945efc (127.23.194.1). New cstate: current_term: 1 leader_uuid: "9f252c5974bb47c4bb8b81906f945efc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f252c5974bb47c4bb8b81906f945efc" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 45471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "565fc27a820b490e871e5b0d5eda1b8b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c68ed834dcd4e01a33561afe8b6bc11" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 34973 } health_report { overall_health: UNKNOWN } } }
I20260808 23:03:38.467043 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.464818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49288 (local address 127.23.194.1:45471)
0808 23:03:38.464882 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:38.464884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:38.464892 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:03:38.464923 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:38.464924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:38.464939 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:38.464996 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:03:38.465126 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.465624 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.466353 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.466532 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.466586 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.466734 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:03:38.466825 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:03:38.466892 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:38.466932 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 23:03:38.466955 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 23:03:38.468734 18852 raft_consensus.cc:1261] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b [term 1 FOLLOWER]: Refusing update from remote peer 9f252c5974bb47c4bb8b81906f945efc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:03:38.468752 18659 raft_consensus.cc:1261] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11 [term 1 FOLLOWER]: Refusing update from remote peer 9f252c5974bb47c4bb8b81906f945efc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:03:38.469094 18960 consensus_queue.cc:1035] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc [LEADER]: Connected to new peer: Peer: permanent_uuid: "565fc27a820b490e871e5b0d5eda1b8b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:03:38.469225 18955 consensus_queue.cc:1035] T bf6d12379e5842caacdf98326a388265 P 9f252c5974bb47c4bb8b81906f945efc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c68ed834dcd4e01a33561afe8b6bc11" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 34973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:03:38.471863 18965 mvcc.cc:205] Tried to move back new op lower bound from 7316398974845337600 to 7316398974768603136. Current Snapshot: MvccSnapshot[applied={T|T < 7316398974845337600}]
I20260808 23:03:38.472011 18964 mvcc.cc:205] Tried to move back new op lower bound from 7316398974845337600 to 7316398974768603136. Current Snapshot: MvccSnapshot[applied={T|T < 7316398974845337600}]
I20260808 23:03:38.473592 18263 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:03:38.473680 18263 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:03:38.474362 18263 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f8bb8484d144f4a9592c0c37d8f699c: Sending DeleteTablet for 3 replicas of tablet bf6d12379e5842caacdf98326a388265
I20260808 23:03:38.474694 18821 tablet_service.cc:1514] Processing DeleteTablet for tablet bf6d12379e5842caacdf98326a388265 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:03:38 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:49708
I20260808 23:03:38.474905 18978 tablet_replica.cc:330] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b: stopping tablet replica
I20260808 23:03:38.474903 18632 tablet_service.cc:1514] Processing DeleteTablet for tablet bf6d12379e5842caacdf98326a388265 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:03:38 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:48564
I20260808 23:03:38.475003 18978 raft_consensus.cc:2229] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:03:38.475102 18978 raft_consensus.cc:2258] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:03:38.475128 18981 tablet_replica.cc:330] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11: stopping tablet replica
I20260808 23:03:38.475206 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 18349
I20260808 23:03:38.475279 18981 raft_consensus.cc:2229] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:03:38.475399 18981 raft_consensus.cc:2258] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:03:38.481246 18978 ts_tablet_manager.cc:1905] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:03:38.483254 18978 ts_tablet_manager.cc:1918] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:03:38.483327 18978 log.cc:1196] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/bf6d12379e5842caacdf98326a388265
I20260808 23:03:38.483562 18978 ts_tablet_manager.cc:1939] T bf6d12379e5842caacdf98326a388265 P 565fc27a820b490e871e5b0d5eda1b8b: Deleting consensus metadata
I20260808 23:03:38.483964 18248 catalog_manager.cc:4841] TS 565fc27a820b490e871e5b0d5eda1b8b (127.23.194.3:38191): tablet bf6d12379e5842caacdf98326a388265 (table test-table [id=da8a4ec24b7e409493f17bbf741ee4e2]) successfully deleted
I20260808 23:03:38.483995 18981 ts_tablet_manager.cc:1905] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:03:38.485766 18981 ts_tablet_manager.cc:1918] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:03:38.485850 18981 log.cc:1196] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/bf6d12379e5842caacdf98326a388265
I20260808 23:03:38.486167 18981 ts_tablet_manager.cc:1939] T bf6d12379e5842caacdf98326a388265 P 9c68ed834dcd4e01a33561afe8b6bc11: Deleting consensus metadata
I20260808 23:03:38.486588 18244 catalog_manager.cc:4841] TS 9c68ed834dcd4e01a33561afe8b6bc11 (127.23.194.2:34973): tablet bf6d12379e5842caacdf98326a388265 (table test-table [id=da8a4ec24b7e409493f17bbf741ee4e2]) successfully deleted
W20260808 23:03:38.489609 18244 connection.cc:535] client connection to 127.23.194.1:45471 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 23:03:38.489712 18244 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 23:03:38.489880 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 18543
I20260808 23:03:38.496938 18937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.496778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:45471 (local address 127.0.0.1:49298)
0808 23:03:38.496846 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:38.496899 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.1:45471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 23:03:38.497092 18244 catalog_manager.cc:4584] TS 9f252c5974bb47c4bb8b81906f945efc (127.23.194.1:45471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bf6d12379e5842caacdf98326a388265: Network error: Client connection negotiation failed: client connection to 127.23.194.1:45471: connect: Connection refused (error 111)
I20260808 23:03:38.502390 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 18730
I20260808 23:03:38.522403 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 18218
2026-08-08T23:03:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2071 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T23:03:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:03:38Z Disabled control of system clock
I20260808 23:03:38.556955 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:38507
--webserver_interface=127.23.194.62
--webserver_port=0
--builtin_ntp_servers=127.23.194.20:42491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:38507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:03:38.636519 18987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:38.636731 18987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:38.636751 18987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:38.637910 18987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:03:38.637949 18987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:38.637961 18987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:38.637972 18987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:03:38.637996 18987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:03:38.639330 18987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:42491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:38507
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:38507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:38.639544 18987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:38.639820 18987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:38.644474 18997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:38.644451 19000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:38.644559 18987 server_base.cc:1034] running on GCE node
W20260808 23:03:38.644443 18995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:38.644910 18987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:38.645182 18987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:38.646324 18987 hybrid_clock.cc:648] HybridClock initialized: now 1786230218646299 us; error 43 us; skew 500 ppm
I20260808 23:03:38.647531 18987 webserver.cc:460] Webserver started at http://127.23.194.62:33037/ using document root <none> and password file <none>
I20260808 23:03:38.647781 18987 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:38.647832 18987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:38.647959 18987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:38.648941 18987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "abfb9f85ade44616ab64b459d17fc663"
format_stamp: "Formatted at 2026-08-08 23:03:38 on dist-test-slave-f0tn"
server_key: "ca287ec529ff2144af37a4e03574ca8c"
server_key_iv: "160a3e3bc0e27dc565d117a76816d40c"
server_key_version: "encryptionkey@0"
I20260808 23:03:38.649258 18987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "abfb9f85ade44616ab64b459d17fc663"
format_stamp: "Formatted at 2026-08-08 23:03:38 on dist-test-slave-f0tn"
server_key: "ca287ec529ff2144af37a4e03574ca8c"
server_key_iv: "160a3e3bc0e27dc565d117a76816d40c"
server_key_version: "encryptionkey@0"
I20260808 23:03:38.650429 18987 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 23:03:38.651072 19012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:38.651211 18987 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:03:38.651292 18987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "abfb9f85ade44616ab64b459d17fc663"
format_stamp: "Formatted at 2026-08-08 23:03:38 on dist-test-slave-f0tn"
server_key: "ca287ec529ff2144af37a4e03574ca8c"
server_key_iv: "160a3e3bc0e27dc565d117a76816d40c"
server_key_version: "encryptionkey@0"
I20260808 23:03:38.651350 18987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:38.661296 18987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:38.661599 18987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:38.661752 18987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:38.665475 18987 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:38507
I20260808 23:03:38.665518 19086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:38507 every 8 connection(s)
I20260808 23:03:38.665838 18987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:03:38.666771 19088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:38.668964 19088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663: Bootstrap starting.
I20260808 23:03:38.669533 19088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:38.669807 19088 log.cc:824] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:38.670495 19088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663: No bootstrap required, opened a new log
I20260808 23:03:38.671092 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 18987
I20260808 23:03:38.671186 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:03:38.671348 24328 external_mini_cluster.cc:1428] Setting key e00254ef03d50b6e851d8eca1f5ee0a6
I20260808 23:03:38.672441 19088 raft_consensus.cc:348] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfb9f85ade44616ab64b459d17fc663" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 38507 } }
I20260808 23:03:38.672588 19088 raft_consensus.cc:374] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:38.672626 19088 raft_consensus.cc:729] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abfb9f85ade44616ab64b459d17fc663, State: Initialized, Role: FOLLOWER
I20260808 23:03:38.672750 19088 consensus_queue.cc:260] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfb9f85ade44616ab64b459d17fc663" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 38507 } }
I20260808 23:03:38.672843 19088 raft_consensus.cc:388] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:03:38.672878 19088 raft_consensus.cc:482] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:03:38.672928 19088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:38.673673 19088 raft_consensus.cc:504] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfb9f85ade44616ab64b459d17fc663" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 38507 } }
I20260808 23:03:38.673794 19088 leader_election.cc:302] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abfb9f85ade44616ab64b459d17fc663; no voters:
I20260808 23:03:38.673969 19088 leader_election.cc:288] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:03:38.674011 19096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:38.674146 19096 raft_consensus.cc:686] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663 [term 1 LEADER]: Becoming Leader. State: Replica: abfb9f85ade44616ab64b459d17fc663, State: Running, Role: LEADER
I20260808 23:03:38.674230 19088 sys_catalog.cc:564] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:03:38.674244 19096 consensus_queue.cc:237] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfb9f85ade44616ab64b459d17fc663" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 38507 } }
I20260808 23:03:38.674679 19099 sys_catalog.cc:455] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abfb9f85ade44616ab64b459d17fc663" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfb9f85ade44616ab64b459d17fc663" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 38507 } } }
I20260808 23:03:38.674778 19099 sys_catalog.cc:458] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:38.674685 19100 sys_catalog.cc:455] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abfb9f85ade44616ab64b459d17fc663. Latest consensus state: current_term: 1 leader_uuid: "abfb9f85ade44616ab64b459d17fc663" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfb9f85ade44616ab64b459d17fc663" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 38507 } } }
I20260808 23:03:38.674867 19100 sys_catalog.cc:458] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:38.675523 19111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:03:38.676066 19111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:03:38.678022 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.671837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58644 (local address 127.23.194.62:38507)
0808 23:03:38.672052 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:38.672059 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:38.672083 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:03:38.672409 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:38.672414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:38.672467 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:38.672583 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:38.672853 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.674605 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.675732 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.676316 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.676450 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.677231 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:38.677236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:38.677236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:38.677479 (+ 243us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:38.677524 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:38.677539 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:38.677617 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:38.677718 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 23:03:38.677807 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":101,"threads_started":1}
I20260808 23:03:38.678141 19111 catalog_manager.cc:1347] Generated new cluster ID: ed024f03b7ee4b2c83750dd8b0a25a38
I20260808 23:03:38.678174 19111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:03:38.683854 19111 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:03:38.684311 19111 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:03:38.688911 19111 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663: Generated new TSK 0
I20260808 23:03:38.689062 19111 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:03:38.692876 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:38507
--builtin_ntp_servers=127.23.194.20:42491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:03:38.775362 19128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:38.775560 19128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:38.775588 19128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:38.776741 19128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:38.776784 19128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:38.776816 19128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:03:38.778235 19128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:42491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:38507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:38.778456 19128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:38.778743 19128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:38.784614 19137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:38.784716 19128 server_base.cc:1034] running on GCE node
W20260808 23:03:38.784614 19140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:38.784860 19136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:38.785095 19128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:38.785338 19128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:38.786516 19128 hybrid_clock.cc:648] HybridClock initialized: now 1786230218786504 us; error 34 us; skew 500 ppm
I20260808 23:03:38.787674 19128 webserver.cc:460] Webserver started at http://127.23.194.1:37243/ using document root <none> and password file <none>
I20260808 23:03:38.787894 19128 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:38.787943 19128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:38.788096 19128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:38.789101 19128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "2d50f2a443d94c30923238e709276d71"
format_stamp: "Formatted at 2026-08-08 23:03:38 on dist-test-slave-f0tn"
server_key: "8c7f633ef96a0f7c2d2b7708db78f36a"
server_key_iv: "98f4191e498c04749e8099f3f17b5e5b"
server_key_version: "encryptionkey@0"
I20260808 23:03:38.789479 19128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "2d50f2a443d94c30923238e709276d71"
format_stamp: "Formatted at 2026-08-08 23:03:38 on dist-test-slave-f0tn"
server_key: "8c7f633ef96a0f7c2d2b7708db78f36a"
server_key_iv: "98f4191e498c04749e8099f3f17b5e5b"
server_key_version: "encryptionkey@0"
I20260808 23:03:38.790733 19128 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:03:38.791440 19150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:38.791602 19128 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:38.791687 19128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "2d50f2a443d94c30923238e709276d71"
format_stamp: "Formatted at 2026-08-08 23:03:38 on dist-test-slave-f0tn"
server_key: "8c7f633ef96a0f7c2d2b7708db78f36a"
server_key_iv: "98f4191e498c04749e8099f3f17b5e5b"
server_key_version: "encryptionkey@0"
I20260808 23:03:38.791756 19128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:38.809953 19128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:38.810242 19128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:38.810374 19128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:38.810588 19128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:38.810902 19128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:38.810937 19128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:38.810966 19128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:38.810983 19128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:38.816444 19128 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:36487
I20260808 23:03:38.816498 19315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:36487 every 8 connection(s)
I20260808 23:03:38.816797 19128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:03:38.818553 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 19128
I20260808 23:03:38.818652 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:03:38.818820 24328 external_mini_cluster.cc:1428] Setting key a6554914d340255607015d22f152d940
I20260808 23:03:38.819736 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:38507
--builtin_ntp_servers=127.23.194.20:42491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 23:03:38.821852 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.817478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:37385 (local address 127.23.194.62:38507)
0808 23:03:38.817536 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:38.817538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:38.817798 (+ 260us) server_negotiation.cc:407] Connection header received
0808 23:03:38.818258 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:38.818259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:38.818270 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:38.818327 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:38.818983 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.819521 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.820496 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.820743 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.820813 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.821616 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:38.821617 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:38.821617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:38.821622 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:38.821636 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:38.821640 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:38.821675 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:38.821772 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 23:03:38.821795 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 23:03:38.822090 19320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.817490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:38507 (local address 127.23.194.1:37385)
0808 23:03:38.817684 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:38.817716 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:38.818181 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:38.818393 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:38.818395 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:38.818404 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:38.818892 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:38.818899 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.819606 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.819607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:38.820417 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:38.820419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.820813 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.820814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:38.820909 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.821458 (+ 549us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:03:38.821459 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:38.821498 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:03:38.821526 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:38.821687 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:38.821691 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:38.821694 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:03:38.821775 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 23:03:38.821883 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":89,"threads_started":1}
I20260808 23:03:38.822556 19316 heartbeater.cc:347] Connected to a master server at 127.23.194.62:38507
I20260808 23:03:38.822649 19316 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:38.822818 19316 heartbeater.cc:511] Master 127.23.194.62:38507 requested a full tablet report, sending...
I20260808 23:03:38.823300 19040 ts_manager.cc:194] Registered new tserver with Master: 2d50f2a443d94c30923238e709276d71 (127.23.194.1:36487)
I20260808 23:03:38.823931 19040 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.194.1:37385
W20260808 23:03:38.900076 19321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:38.900274 19321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:38.900293 19321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:38.901422 19321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:38.901459 19321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:38.901484 19321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:03:38.902812 19321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:42491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:38507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:38.903020 19321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:38.903296 19321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:38.908016 19328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:38.908044 19329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:38.908048 19331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:38.908078 19321 server_base.cc:1034] running on GCE node
I20260808 23:03:38.908375 19321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:38.908607 19321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:38.909737 19321 hybrid_clock.cc:648] HybridClock initialized: now 1786230218909727 us; error 29 us; skew 500 ppm
I20260808 23:03:38.910840 19321 webserver.cc:460] Webserver started at http://127.23.194.2:37053/ using document root <none> and password file <none>
I20260808 23:03:38.911051 19321 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:38.911099 19321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:38.911240 19321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:38.912209 19321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "29eff7396d8a43fbbfbc63cc7da40e99"
format_stamp: "Formatted at 2026-08-08 23:03:38 on dist-test-slave-f0tn"
server_key: "7f35619d094ae7a0730cf2df1b687646"
server_key_iv: "f21d9081a1bdc8679f8f52359d02ae19"
server_key_version: "encryptionkey@0"
I20260808 23:03:38.912587 19321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "29eff7396d8a43fbbfbc63cc7da40e99"
format_stamp: "Formatted at 2026-08-08 23:03:38 on dist-test-slave-f0tn"
server_key: "7f35619d094ae7a0730cf2df1b687646"
server_key_iv: "f21d9081a1bdc8679f8f52359d02ae19"
server_key_version: "encryptionkey@0"
I20260808 23:03:38.913748 19321 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:03:38.914381 19339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:38.914525 19321 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:38.914600 19321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "29eff7396d8a43fbbfbc63cc7da40e99"
format_stamp: "Formatted at 2026-08-08 23:03:38 on dist-test-slave-f0tn"
server_key: "7f35619d094ae7a0730cf2df1b687646"
server_key_iv: "f21d9081a1bdc8679f8f52359d02ae19"
server_key_version: "encryptionkey@0"
I20260808 23:03:38.914662 19321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:38.927670 19321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:38.927901 19321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:38.928016 19321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:38.928205 19321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:38.928489 19321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:38.928529 19321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:38.928550 19321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:38.928572 19321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:38.934216 19321 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:37383
I20260808 23:03:38.934254 19508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:37383 every 8 connection(s)
I20260808 23:03:38.934525 19321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:03:38.935184 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 19321
I20260808 23:03:38.935293 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:03:38.935473 24328 external_mini_cluster.cc:1428] Setting key 551f4bb72360cd8a5926d8f531425c6c
I20260808 23:03:38.936369 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:38507
--builtin_ntp_servers=127.23.194.20:42491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 23:03:38.939783 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.935218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:36887 (local address 127.23.194.62:38507)
0808 23:03:38.935277 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:38.935279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:38.935499 (+ 220us) server_negotiation.cc:407] Connection header received
0808 23:03:38.936048 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:38.936050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:38.936062 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:38.936112 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:38.936802 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.937289 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.938517 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.938715 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.938772 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.939550 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:38.939551 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:38.939551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:38.939556 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:38.939573 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:38.939577 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:38.939632 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:38.939680 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 23:03:38.939707 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 23:03:38.939982 19511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:38.935213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:38507 (local address 127.23.194.2:36887)
0808 23:03:38.935397 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:38.935427 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:38.935948 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:38.936183 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:38.936187 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:38.936197 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:38.936715 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:38.936720 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.937368 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.937370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:38.938428 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:38.938431 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:38.938793 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:38.938794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:38.938940 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:38.939428 (+ 488us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:03:38.939428 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:38.939455 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:03:38.939474 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:38.939634 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:38.939636 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:38.939639 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:03:38.939681 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 23:03:38.939752 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260808 23:03:38.940415 19509 heartbeater.cc:347] Connected to a master server at 127.23.194.62:38507
I20260808 23:03:38.940498 19509 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:38.940659 19509 heartbeater.cc:511] Master 127.23.194.62:38507 requested a full tablet report, sending...
I20260808 23:03:38.940999 19040 ts_manager.cc:194] Registered new tserver with Master: 29eff7396d8a43fbbfbc63cc7da40e99 (127.23.194.2:37383)
I20260808 23:03:38.941507 19040 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.194.2:36887
W20260808 23:03:39.018244 19512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:39.018427 19512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:39.018447 19512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:39.019501 19512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:39.019539 19512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:39.019562 19512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:03:39.020817 19512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:42491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:38507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:39.020985 19512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:39.021212 19512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:39.025929 19517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:39.025975 19518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:39.025987 19521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:39.026023 19512 server_base.cc:1034] running on GCE node
I20260808 23:03:39.026350 19512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:39.026552 19512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:39.027693 19512 hybrid_clock.cc:648] HybridClock initialized: now 1786230219027679 us; error 29 us; skew 500 ppm
I20260808 23:03:39.028887 19512 webserver.cc:460] Webserver started at http://127.23.194.3:38379/ using document root <none> and password file <none>
I20260808 23:03:39.029093 19512 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:39.029136 19512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:39.029256 19512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:39.030333 19512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "7dd14896ab814b37b2f26ddc815fe312"
format_stamp: "Formatted at 2026-08-08 23:03:39 on dist-test-slave-f0tn"
server_key: "bb14d29a706b35bf28b0c95095bbacc5"
server_key_iv: "4bf1722c35f556ca73d9a9f1d2f19005"
server_key_version: "encryptionkey@0"
I20260808 23:03:39.030676 19512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "7dd14896ab814b37b2f26ddc815fe312"
format_stamp: "Formatted at 2026-08-08 23:03:39 on dist-test-slave-f0tn"
server_key: "bb14d29a706b35bf28b0c95095bbacc5"
server_key_iv: "4bf1722c35f556ca73d9a9f1d2f19005"
server_key_version: "encryptionkey@0"
I20260808 23:03:39.031862 19512 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:03:39.032531 19527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:39.032656 19512 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:39.032732 19512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "7dd14896ab814b37b2f26ddc815fe312"
format_stamp: "Formatted at 2026-08-08 23:03:39 on dist-test-slave-f0tn"
server_key: "bb14d29a706b35bf28b0c95095bbacc5"
server_key_iv: "4bf1722c35f556ca73d9a9f1d2f19005"
server_key_version: "encryptionkey@0"
I20260808 23:03:39.032796 19512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:39.043793 19512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:39.044029 19512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:39.044143 19512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:39.044339 19512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:39.044636 19512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:39.044669 19512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:39.044688 19512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:39.044710 19512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:39.050189 19512 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:43135
I20260808 23:03:39.050243 19689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:43135 every 8 connection(s)
I20260808 23:03:39.050525 19512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 23:03:39.052230 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 19512
I20260808 23:03:39.052342 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:03:39.052543 24328 external_mini_cluster.cc:1428] Setting key 913ef8b05a411f95029ae37abf9186ef
I20260808 23:03:39.055575 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:39.051128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:42435 (local address 127.23.194.62:38507)
0808 23:03:39.051182 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:39.051184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:39.051391 (+ 207us) server_negotiation.cc:407] Connection header received
0808 23:03:39.051856 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:39.051857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:39.051871 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:39.051929 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:39.052746 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.053489 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.054441 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.054651 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.054716 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:39.055404 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:39.055405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:39.055405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:39.055409 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:39.055419 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:39.055422 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:39.055452 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:39.055499 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 23:03:39.055519 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 23:03:39.055754 19692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:39.051124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:38507 (local address 127.23.194.3:42435)
0808 23:03:39.051283 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:39.051315 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:39.051766 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:39.051990 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:39.051994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:39.052001 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:39.052626 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:39.052634 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.053569 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.053570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:39.054355 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:39.054358 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.054718 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.054718 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:39.054821 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:39.055297 (+ 476us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:03:39.055297 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:39.055323 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:03:39.055343 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:39.055455 (+ 112us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:39.055457 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:39.055459 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 23:03:39.055501 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 23:03:39.055578 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":65,"threads_started":1}
I20260808 23:03:39.056156 19690 heartbeater.cc:347] Connected to a master server at 127.23.194.62:38507
I20260808 23:03:39.056237 19690 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:39.056371 19690 heartbeater.cc:511] Master 127.23.194.62:38507 requested a full tablet report, sending...
I20260808 23:03:39.056684 19040 ts_manager.cc:194] Registered new tserver with Master: 7dd14896ab814b37b2f26ddc815fe312 (127.23.194.3:43135)
I20260808 23:03:39.057000 19040 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.194.3:42435
I20260808 23:03:39.064028 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:03:39.067895 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:39.065270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58660 (local address 127.23.194.62:38507)
0808 23:03:39.065327 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:39.065329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:39.065431 (+ 102us) server_negotiation.cc:407] Connection header received
0808 23:03:39.065499 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:39.065500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:39.065511 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:39.065553 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:39.065698 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.066161 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.066876 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.067060 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.067110 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:39.067684 (+ 574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:39.067685 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:39.067686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:39.067690 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:39.067702 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:39.067706 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:39.067743 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:39.067802 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 23:03:39.067824 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 23:03:39.069317 19040 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:03:39.070802 19040 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:03:39.076949 19703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:39.072767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52922 (local address 127.23.194.1:36487)
0808 23:03:39.073176 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:39.073181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:39.073192 (+ 11us) server_negotiation.cc:407] Connection header received
0808 23:03:39.073235 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:39.073237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:39.073253 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:39.073356 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:39.073613 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.074711 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.075927 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.076584 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.076643 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:39.076714 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:39.076829 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 23:03:39.076870 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":93,"threads_started":1}
I20260808 23:03:39.076956 19706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:39.072675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:36487 (local address 127.0.0.1:52922)
0808 23:03:39.072992 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:39.073001 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:39.073039 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:39.073363 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:39.073366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:39.073368 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:39.073560 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:39.073561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.074790 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.074791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:39.075850 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:39.075853 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.076648 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.076648 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:39.076718 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:39.076733 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 23:03:39.076805 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 23:03:39.076861 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":21,"thread_start_us":88,"threads_started":1}
I20260808 23:03:39.077196 19707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:39.072964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59884 (local address 127.23.194.3:43135)
0808 23:03:39.073146 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:39.073151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:39.073261 (+ 110us) server_negotiation.cc:407] Connection header received
0808 23:03:39.073320 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:39.073322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:39.073335 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:39.073534 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:39.073716 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.074807 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.075843 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.076541 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.076615 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:39.076705 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:39.077081 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 23:03:39.077120 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":90,"threads_started":1}
I20260808 23:03:39.076956 19699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:39.072869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:43135 (local address 127.0.0.1:59884)
0808 23:03:39.073193 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:39.073203 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:39.073259 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:39.073593 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:39.073594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:39.073597 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:39.073640 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:39.073642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.074862 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.074863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:39.075773 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:39.075775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.076619 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.076619 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:39.076718 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:39.076733 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 23:03:39.076805 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 23:03:39.076861 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":75,"threads_started":1}
I20260808 23:03:39.077805 19701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:39.072675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:37383 (local address 127.0.0.1:54022)
0808 23:03:39.072827 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:39.072867 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:39.072929 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:39.073409 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:39.073410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:39.073411 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:39.073464 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:39.073466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.074565 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.074566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:39.075850 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:39.075853 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.077605 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.077606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:39.077668 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:39.077673 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:03:39.077701 (+ 28us) client_negotiation.cc:240] Negotiation successful
0808 23:03:39.077740 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260808 23:03:39.077905 19225 tablet_service.cc:1467] Processing CreateTablet for tablet 0a5b157ade774db2a5c45304927ed9e7 (DEFAULT_TABLE table=test-table [id=69ce51b2d40b4443aca2bc17b419fc59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:39.078166 19709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:39.073016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54022 (local address 127.23.194.2:37383)
0808 23:03:39.073183 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:39.073186 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:39.073195 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:03:39.073231 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:39.073233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:39.073246 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:39.073353 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:39.073531 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.074491 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.076749 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.077506 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.077912 (+ 406us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:39.078007 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:39.078051 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 23:03:39.078089 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260808 23:03:39.078321 19225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a5b157ade774db2a5c45304927ed9e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:39.078213 19594 tablet_service.cc:1467] Processing CreateTablet for tablet 0a5b157ade774db2a5c45304927ed9e7 (DEFAULT_TABLE table=test-table [id=69ce51b2d40b4443aca2bc17b419fc59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:39.078547 19594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a5b157ade774db2a5c45304927ed9e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:39.078755 19411 tablet_service.cc:1467] Processing CreateTablet for tablet 0a5b157ade774db2a5c45304927ed9e7 (DEFAULT_TABLE table=test-table [id=69ce51b2d40b4443aca2bc17b419fc59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:39.079087 19411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a5b157ade774db2a5c45304927ed9e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:39.081028 19710 tablet_bootstrap.cc:492] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312: Bootstrap starting.
I20260808 23:03:39.081693 19710 tablet_bootstrap.cc:654] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:39.081954 19711 tablet_bootstrap.cc:492] T 0a5b157ade774db2a5c45304927ed9e7 P 2d50f2a443d94c30923238e709276d71: Bootstrap starting.
I20260808 23:03:39.082028 19710 log.cc:824] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:39.082326 19712 tablet_bootstrap.cc:492] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99: Bootstrap starting.
I20260808 23:03:39.082641 19711 tablet_bootstrap.cc:654] T 0a5b157ade774db2a5c45304927ed9e7 P 2d50f2a443d94c30923238e709276d71: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:39.082778 19710 tablet_bootstrap.cc:492] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312: No bootstrap required, opened a new log
I20260808 23:03:39.082849 19710 ts_tablet_manager.cc:1397] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 23:03:39.082886 19712 tablet_bootstrap.cc:654] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:39.082957 19711 log.cc:824] T 0a5b157ade774db2a5c45304927ed9e7 P 2d50f2a443d94c30923238e709276d71: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:39.083186 19712 log.cc:824] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:39.083756 19711 tablet_bootstrap.cc:492] T 0a5b157ade774db2a5c45304927ed9e7 P 2d50f2a443d94c30923238e709276d71: No bootstrap required, opened a new log
I20260808 23:03:39.083822 19711 ts_tablet_manager.cc:1397] T 0a5b157ade774db2a5c45304927ed9e7 P 2d50f2a443d94c30923238e709276d71: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:03:39.085250 19710 raft_consensus.cc:348] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d50f2a443d94c30923238e709276d71" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36487 } } peers { permanent_uuid: "29eff7396d8a43fbbfbc63cc7da40e99" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 37383 } } peers { permanent_uuid: "7dd14896ab814b37b2f26ddc815fe312" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43135 } }
I20260808 23:03:39.085443 19710 raft_consensus.cc:374] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:39.085487 19710 raft_consensus.cc:729] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd14896ab814b37b2f26ddc815fe312, State: Initialized, Role: FOLLOWER
I20260808 23:03:39.085614 19710 consensus_queue.cc:260] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d50f2a443d94c30923238e709276d71" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36487 } } peers { permanent_uuid: "29eff7396d8a43fbbfbc63cc7da40e99" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 37383 } } peers { permanent_uuid: "7dd14896ab814b37b2f26ddc815fe312" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43135 } }
I20260808 23:03:39.085875 19710 ts_tablet_manager.cc:1428] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 23:03:39.085917 19712 tablet_bootstrap.cc:492] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99: No bootstrap required, opened a new log
I20260808 23:03:39.085973 19712 ts_tablet_manager.cc:1397] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 23:03:39.085989 19690 heartbeater.cc:503] Master 127.23.194.62:38507 was elected leader, sending a full tablet report...
I20260808 23:03:39.086171 19711 raft_consensus.cc:348] T 0a5b157ade774db2a5c45304927ed9e7 P 2d50f2a443d94c30923238e709276d71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d50f2a443d94c30923238e709276d71" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36487 } } peers { permanent_uuid: "29eff7396d8a43fbbfbc63cc7da40e99" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 37383 } } peers { permanent_uuid: "7dd14896ab814b37b2f26ddc815fe312" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43135 } }
I20260808 23:03:39.086318 19711 raft_consensus.cc:374] T 0a5b157ade774db2a5c45304927ed9e7 P 2d50f2a443d94c30923238e709276d71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:39.086355 19711 raft_consensus.cc:729] T 0a5b157ade774db2a5c45304927ed9e7 P 2d50f2a443d94c30923238e709276d71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d50f2a443d94c30923238e709276d71, State: Initialized, Role: FOLLOWER
I20260808 23:03:39.086457 19711 consensus_queue.cc:260] T 0a5b157ade774db2a5c45304927ed9e7 P 2d50f2a443d94c30923238e709276d71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d50f2a443d94c30923238e709276d71" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36487 } } peers { permanent_uuid: "29eff7396d8a43fbbfbc63cc7da40e99" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 37383 } } peers { permanent_uuid: "7dd14896ab814b37b2f26ddc815fe312" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43135 } }
I20260808 23:03:39.086695 19711 ts_tablet_manager.cc:1428] T 0a5b157ade774db2a5c45304927ed9e7 P 2d50f2a443d94c30923238e709276d71: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 23:03:39.086719 19316 heartbeater.cc:503] Master 127.23.194.62:38507 was elected leader, sending a full tablet report...
I20260808 23:03:39.088181 19712 raft_consensus.cc:348] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d50f2a443d94c30923238e709276d71" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36487 } } peers { permanent_uuid: "29eff7396d8a43fbbfbc63cc7da40e99" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 37383 } } peers { permanent_uuid: "7dd14896ab814b37b2f26ddc815fe312" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43135 } }
I20260808 23:03:39.088296 19712 raft_consensus.cc:374] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:39.088315 19712 raft_consensus.cc:729] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29eff7396d8a43fbbfbc63cc7da40e99, State: Initialized, Role: FOLLOWER
I20260808 23:03:39.088416 19712 consensus_queue.cc:260] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d50f2a443d94c30923238e709276d71" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36487 } } peers { permanent_uuid: "29eff7396d8a43fbbfbc63cc7da40e99" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 37383 } } peers { permanent_uuid: "7dd14896ab814b37b2f26ddc815fe312" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43135 } }
I20260808 23:03:39.088632 19712 ts_tablet_manager.cc:1428] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 23:03:39.088697 19509 heartbeater.cc:503] Master 127.23.194.62:38507 was elected leader, sending a full tablet report...
I20260808 23:03:39.111860 19718 raft_consensus.cc:482] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:03:39.111950 19718 raft_consensus.cc:504] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d50f2a443d94c30923238e709276d71" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36487 } } peers { permanent_uuid: "29eff7396d8a43fbbfbc63cc7da40e99" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 37383 } } peers { permanent_uuid: "7dd14896ab814b37b2f26ddc815fe312" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43135 } }
I20260808 23:03:39.112242 19718 leader_election.cc:288] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d50f2a443d94c30923238e709276d71 (127.23.194.1:36487), 29eff7396d8a43fbbfbc63cc7da40e99 (127.23.194.2:37383)
I20260808 23:03:39.115993 19709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:39.112503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:41037 (local address 127.23.194.2:37383)
0808 23:03:39.112563 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:39.112565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:39.112775 (+ 210us) server_negotiation.cc:407] Connection header received
0808 23:03:39.112797 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:39.112798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:39.112805 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:39.112846 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:39.112987 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.113690 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.115063 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.115617 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.115690 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:39.115698 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:39.115890 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 23:03:39.115926 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 23:03:39.115994 19703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:39.112516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:50331 (local address 127.23.194.1:36487)
0808 23:03:39.112555 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:39.112557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:39.112568 (+ 11us) server_negotiation.cc:407] Connection header received
0808 23:03:39.112626 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:39.112626 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:39.112636 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:39.112689 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:39.112869 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.113632 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.115063 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.115617 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.115693 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:39.115698 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:39.115890 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 23:03:39.115926 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 23:03:39.116016 19692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:39.112443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:36487 (local address 127.23.194.3:50331)
0808 23:03:39.112504 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:39.112516 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:39.112563 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:39.112716 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:39.112717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:39.112718 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:39.112792 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:39.112793 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.113727 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.113728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:39.114984 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:39.114986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.115713 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.115714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:39.115814 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:39.115820 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 23:03:39.115880 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 23:03:39.115925 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":39}
I20260808 23:03:39.116029 19721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:39.112443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:37383 (local address 127.23.194.3:41037)
0808 23:03:39.112678 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:39.112688 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:39.112741 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:39.112862 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:39.112863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:39.112864 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:39.112928 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:39.112929 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.113755 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.113756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:39.114979 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:39.114982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.115726 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.115727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:39.115814 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:39.115821 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 23:03:39.115880 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 23:03:39.115924 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":25,"thread_start_us":142,"threads_started":1}
I20260808 23:03:39.116338 19254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a5b157ade774db2a5c45304927ed9e7" candidate_uuid: "7dd14896ab814b37b2f26ddc815fe312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d50f2a443d94c30923238e709276d71" is_pre_election: true
I20260808 23:03:39.116338 19437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a5b157ade774db2a5c45304927ed9e7" candidate_uuid: "7dd14896ab814b37b2f26ddc815fe312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29eff7396d8a43fbbfbc63cc7da40e99" is_pre_election: true
I20260808 23:03:39.116480 19254 raft_consensus.cc:2454] T 0a5b157ade774db2a5c45304927ed9e7 P 2d50f2a443d94c30923238e709276d71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd14896ab814b37b2f26ddc815fe312 in term 0.
I20260808 23:03:39.116501 19437 raft_consensus.cc:2454] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd14896ab814b37b2f26ddc815fe312 in term 0.
I20260808 23:03:39.116706 19528 leader_election.cc:302] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29eff7396d8a43fbbfbc63cc7da40e99, 7dd14896ab814b37b2f26ddc815fe312; no voters:
I20260808 23:03:39.116837 19718 raft_consensus.cc:2781] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:03:39.116910 19718 raft_consensus.cc:482] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:03:39.116927 19718 raft_consensus.cc:3037] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:39.117722 19718 raft_consensus.cc:504] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d50f2a443d94c30923238e709276d71" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36487 } } peers { permanent_uuid: "29eff7396d8a43fbbfbc63cc7da40e99" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 37383 } } peers { permanent_uuid: "7dd14896ab814b37b2f26ddc815fe312" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43135 } }
I20260808 23:03:39.117839 19718 leader_election.cc:288] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [CANDIDATE]: Term 1 election: Requested vote from peers 2d50f2a443d94c30923238e709276d71 (127.23.194.1:36487), 29eff7396d8a43fbbfbc63cc7da40e99 (127.23.194.2:37383)
I20260808 23:03:39.117972 19254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a5b157ade774db2a5c45304927ed9e7" candidate_uuid: "7dd14896ab814b37b2f26ddc815fe312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d50f2a443d94c30923238e709276d71"
I20260808 23:03:39.117972 19437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a5b157ade774db2a5c45304927ed9e7" candidate_uuid: "7dd14896ab814b37b2f26ddc815fe312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29eff7396d8a43fbbfbc63cc7da40e99"
I20260808 23:03:39.118044 19437 raft_consensus.cc:3037] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:39.118044 19254 raft_consensus.cc:3037] T 0a5b157ade774db2a5c45304927ed9e7 P 2d50f2a443d94c30923238e709276d71 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:39.118886 19437 raft_consensus.cc:2454] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd14896ab814b37b2f26ddc815fe312 in term 1.
I20260808 23:03:39.118886 19254 raft_consensus.cc:2454] T 0a5b157ade774db2a5c45304927ed9e7 P 2d50f2a443d94c30923238e709276d71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd14896ab814b37b2f26ddc815fe312 in term 1.
I20260808 23:03:39.119045 19530 leader_election.cc:302] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d50f2a443d94c30923238e709276d71, 7dd14896ab814b37b2f26ddc815fe312; no voters:
I20260808 23:03:39.119153 19718 raft_consensus.cc:2781] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:39.119287 19718 raft_consensus.cc:686] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [term 1 LEADER]: Becoming Leader. State: Replica: 7dd14896ab814b37b2f26ddc815fe312, State: Running, Role: LEADER
I20260808 23:03:39.119371 19718 consensus_queue.cc:237] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d50f2a443d94c30923238e709276d71" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36487 } } peers { permanent_uuid: "29eff7396d8a43fbbfbc63cc7da40e99" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 37383 } } peers { permanent_uuid: "7dd14896ab814b37b2f26ddc815fe312" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43135 } }
I20260808 23:03:39.120117 19039 catalog_manager.cc:5495] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dd14896ab814b37b2f26ddc815fe312 (127.23.194.3). New cstate: current_term: 1 leader_uuid: "7dd14896ab814b37b2f26ddc815fe312" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d50f2a443d94c30923238e709276d71" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29eff7396d8a43fbbfbc63cc7da40e99" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 37383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd14896ab814b37b2f26ddc815fe312" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43135 } health_report { overall_health: HEALTHY } } }
I20260808 23:03:39.124467 19707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:39.122604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59894 (local address 127.23.194.3:43135)
0808 23:03:39.122645 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:39.122647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:39.122660 (+ 13us) server_negotiation.cc:407] Connection header received
0808 23:03:39.122719 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:39.122720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:39.122734 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:39.122787 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:03:39.122864 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.123289 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.123946 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:39.124110 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:39.124149 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:39.124245 (+ 96us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:03:39.124331 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:03:39.124380 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:39.124393 (+ 13us) server_negotiation.cc:299] Negotiation successful
0808 23:03:39.124405 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 23:03:39.125973 19437 raft_consensus.cc:1261] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99 [term 1 FOLLOWER]: Refusing update from remote peer 7dd14896ab814b37b2f26ddc815fe312: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:03:39.125973 19254 raft_consensus.cc:1261] T 0a5b157ade774db2a5c45304927ed9e7 P 2d50f2a443d94c30923238e709276d71 [term 1 FOLLOWER]: Refusing update from remote peer 7dd14896ab814b37b2f26ddc815fe312: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:03:39.126272 19722 consensus_queue.cc:1035] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29eff7396d8a43fbbfbc63cc7da40e99" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 37383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:03:39.126390 19718 consensus_queue.cc:1035] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d50f2a443d94c30923238e709276d71" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:03:39.128721 19726 mvcc.cc:205] Tried to move back new op lower bound from 7316398977537912832 to 7316398977513635840. Current Snapshot: MvccSnapshot[applied={T|T < 7316398977537912832}]
I20260808 23:03:39.130134 19727 mvcc.cc:205] Tried to move back new op lower bound from 7316398977537912832 to 7316398977513635840. Current Snapshot: MvccSnapshot[applied={T|T < 7316398977537912832 or (T in {7316398977537912832})}]
I20260808 23:03:39.130437 19039 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:03:39.130522 19039 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:03:39.131183 19039 catalog_manager.cc:5782] T 00000000000000000000000000000000 P abfb9f85ade44616ab64b459d17fc663: Sending DeleteTablet for 3 replicas of tablet 0a5b157ade774db2a5c45304927ed9e7
I20260808 23:03:39.131516 19225 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a5b157ade774db2a5c45304927ed9e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:03:39 UTC) from {username='slave'} at 127.0.0.1:52922
I20260808 23:03:39.131716 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 19128
I20260808 23:03:39.131780 19594 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a5b157ade774db2a5c45304927ed9e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:03:39 UTC) from {username='slave'} at 127.0.0.1:59884
I20260808 23:03:39.137265 19411 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a5b157ade774db2a5c45304927ed9e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:03:39 UTC) from {username='slave'} at 127.0.0.1:54022
I20260808 23:03:39.140761 19737 tablet_replica.cc:330] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312: stopping tablet replica
I20260808 23:03:39.140902 19737 raft_consensus.cc:2229] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:03:39.141036 19737 raft_consensus.cc:2258] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:03:39.142609 19737 ts_tablet_manager.cc:1905] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:03:39.144426 19737 ts_tablet_manager.cc:1918] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:03:39.144510 19737 log.cc:1196] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/0a5b157ade774db2a5c45304927ed9e7
I20260808 23:03:39.144841 19737 ts_tablet_manager.cc:1939] T 0a5b157ade774db2a5c45304927ed9e7 P 7dd14896ab814b37b2f26ddc815fe312: Deleting consensus metadata
I20260808 23:03:39.145335 19013 catalog_manager.cc:4841] TS 7dd14896ab814b37b2f26ddc815fe312 (127.23.194.3:43135): tablet 0a5b157ade774db2a5c45304927ed9e7 (table test-table [id=69ce51b2d40b4443aca2bc17b419fc59]) successfully deleted
I20260808 23:03:39.147181 19738 tablet_replica.cc:330] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99: stopping tablet replica
I20260808 23:03:39.147310 19738 raft_consensus.cc:2229] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:03:39.147431 19738 raft_consensus.cc:2258] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:03:39.147905 19738 ts_tablet_manager.cc:1905] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:03:39.149137 19738 ts_tablet_manager.cc:1918] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:03:39.149214 19738 log.cc:1196] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/0a5b157ade774db2a5c45304927ed9e7
I20260808 23:03:39.149446 19738 ts_tablet_manager.cc:1939] T 0a5b157ade774db2a5c45304927ed9e7 P 29eff7396d8a43fbbfbc63cc7da40e99: Deleting consensus metadata
I20260808 23:03:39.149852 19013 catalog_manager.cc:4841] TS 29eff7396d8a43fbbfbc63cc7da40e99 (127.23.194.2:37383): tablet 0a5b157ade774db2a5c45304927ed9e7 (table test-table [id=69ce51b2d40b4443aca2bc17b419fc59]) successfully deleted
W20260808 23:03:39.152622 19015 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.194.1:36487 (error 108)
I20260808 23:03:39.152882 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 19321
I20260808 23:03:39.155999 19701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:39.152958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:36487 (local address 127.0.0.1:52928)
0808 23:03:39.155906 (+ 2948us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:39.155961 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.1:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2921}
W20260808 23:03:39.159215 19015 catalog_manager.cc:4584] TS 2d50f2a443d94c30923238e709276d71 (127.23.194.1:36487): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a5b157ade774db2a5c45304927ed9e7: Network error: Client connection negotiation failed: client connection to 127.23.194.1:36487: connect: Connection refused (error 111)
I20260808 23:03:39.162712 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 19512
I20260808 23:03:39.175390 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 18987
2026-08-08T23:03:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (651 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:03:39 dist-test-slave-f0tn krb5kdc[19743](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:03:39 dist-test-slave-f0tn krb5kdc[19743](info): set up 2 sockets
Aug 08 23:03:39 dist-test-slave-f0tn krb5kdc[19743](info): commencing operation
krb5kdc: starting...
W20260808 23:03:40.308780 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:03:40 dist-test-slave-f0tn krb5kdc[19743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:03:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:03:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:03:40.419521 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:39557
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:42137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:39557
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:40.499084 19765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:40.499298 19765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:40.499328 19765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:40.500461 19765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:03:40.500500 19765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:40.500517 19765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:40.500533 19765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:03:40.500548 19765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:03:40.502038 19765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:42137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:39557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:39557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:40.502280 19765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:40.502475 19765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:40.507192 19772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:40.507221 19774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:40.507300 19765 server_base.cc:1034] running on GCE node
W20260808 23:03:40.507221 19776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:40.507659 19765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:40.507948 19765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:40.509088 19765 hybrid_clock.cc:648] HybridClock initialized: now 1786230220509070 us; error 33 us; skew 500 ppm
Aug 08 23:03:40 dist-test-slave-f0tn krb5kdc[19743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230220, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:40.515936 19765 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:03:40.516501 19765 webserver.cc:460] Webserver started at http://127.23.194.62:32935/ using document root <none> and password file <none>
I20260808 23:03:40.516829 19765 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:40.516893 19765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:40.517046 19765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:40.518298 19765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "cf550ee7dd554169b1e7b79318d6d479"
format_stamp: "Formatted at 2026-08-08 23:03:40 on dist-test-slave-f0tn"
server_key: "8b673093fa9ee4f4fe88dc54318f897f"
server_key_iv: "672efbfbefa9909c12a647b75688983c"
server_key_version: "encryptionkey@0"
I20260808 23:03:40.518641 19765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "cf550ee7dd554169b1e7b79318d6d479"
format_stamp: "Formatted at 2026-08-08 23:03:40 on dist-test-slave-f0tn"
server_key: "8b673093fa9ee4f4fe88dc54318f897f"
server_key_iv: "672efbfbefa9909c12a647b75688983c"
server_key_version: "encryptionkey@0"
I20260808 23:03:40.519953 19765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 23:03:40.520587 19787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:40.520742 19765 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:03:40.520816 19765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "cf550ee7dd554169b1e7b79318d6d479"
format_stamp: "Formatted at 2026-08-08 23:03:40 on dist-test-slave-f0tn"
server_key: "8b673093fa9ee4f4fe88dc54318f897f"
server_key_iv: "672efbfbefa9909c12a647b75688983c"
server_key_version: "encryptionkey@0"
I20260808 23:03:40.520882 19765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:40.535688 19765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:40.537576 19765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:40.537720 19765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:40.541447 19765 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:39557
I20260808 23:03:40.541504 19869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:39557 every 8 connection(s)
I20260808 23:03:40.541841 19765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:03:40.542690 19870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:40.543967 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 19765
I20260808 23:03:40.544102 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:03:40.544291 24328 external_mini_cluster.cc:1428] Setting key a14d1ab9d0b4ceded4a2f67e1ba5a355
I20260808 23:03:40.545370 19870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479: Bootstrap starting.
I20260808 23:03:40.545975 19870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:40.546250 19870 log.cc:824] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:40.546912 19870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479: No bootstrap required, opened a new log
I20260808 23:03:40.548694 19870 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf550ee7dd554169b1e7b79318d6d479" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39557 } }
I20260808 23:03:40.548851 19870 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:40.548895 19870 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf550ee7dd554169b1e7b79318d6d479, State: Initialized, Role: FOLLOWER
I20260808 23:03:40.549032 19870 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf550ee7dd554169b1e7b79318d6d479" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39557 } }
I20260808 23:03:40.549120 19870 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:03:40.549157 19870 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:03:40.549206 19870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:40.549778 19870 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf550ee7dd554169b1e7b79318d6d479" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39557 } }
I20260808 23:03:40.549875 19870 leader_election.cc:302] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf550ee7dd554169b1e7b79318d6d479; no voters:
I20260808 23:03:40.550031 19870 leader_election.cc:288] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:03:40.550077 19876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:40.550225 19876 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479 [term 1 LEADER]: Becoming Leader. State: Replica: cf550ee7dd554169b1e7b79318d6d479, State: Running, Role: LEADER
I20260808 23:03:40.550282 19870 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:03:40.550406 19876 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf550ee7dd554169b1e7b79318d6d479" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39557 } }
Aug 08 23:03:40 dist-test-slave-f0tn krb5kdc[19743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:03:40.550849 19879 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf550ee7dd554169b1e7b79318d6d479. Latest consensus state: current_term: 1 leader_uuid: "cf550ee7dd554169b1e7b79318d6d479" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf550ee7dd554169b1e7b79318d6d479" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39557 } } }
I20260808 23:03:40.550925 19879 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:40.551157 19880 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf550ee7dd554169b1e7b79318d6d479" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf550ee7dd554169b1e7b79318d6d479" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39557 } } }
I20260808 23:03:40.551631 19880 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479 [sys.catalog]: This master's current role is: LEADER
W20260808 23:03:40.552145 19904 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:03:40.552189 19904 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:03:40.552284 19888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:03:40.552435 19888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:03:40.553794 19888 catalog_manager.cc:1347] Generated new cluster ID: d85ab1b8af0547289eb796507e8cef59
I20260808 23:03:40.553845 19888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:03:40.555161 19874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:40.544806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55836 (local address 127.23.194.62:39557)
0808 23:03:40.544984 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:40.544992 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:40.545016 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:03:40.545446 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:40.545454 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:40.545517 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:40.545667 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:40.546281 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:40.547342 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:40.548070 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:40.548469 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:40.548562 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:40.551126 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:40.551130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:40.551130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:40.551156 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:40.553123 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:40.553514 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:40.553517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:40.553518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:40.553638 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:40.553929 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:40.553930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:40.553931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:40.554237 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:40.554380 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:40.554818 (+ 438us) server_negotiation.cc:299] Negotiation successful
0808 23:03:40.554957 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":81,"threads_started":1}
I20260808 23:03:40.561451 19888 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:03:40.561982 19888 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:03:40.564963 19888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479: Generated new TSK 0
I20260808 23:03:40.565124 19888 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
I20260808 23:03:40.590294 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:39557
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:42137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:40.671875 19911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:40.672087 19911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:40.672112 19911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:40.673185 19911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:40.673221 19911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:40.673247 19911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:03:40.674731 19911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:42137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:39557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:40.674916 19911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:40.675110 19911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:40.679996 19918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:40.680037 19921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:40.680075 19911 server_base.cc:1034] running on GCE node
W20260808 23:03:40.680037 19916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:40.680424 19911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:40.680693 19911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:40.681843 19911 hybrid_clock.cc:648] HybridClock initialized: now 1786230220681828 us; error 30 us; skew 500 ppm
Aug 08 23:03:40 dist-test-slave-f0tn krb5kdc[19743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230220, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:40.684337 19911 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 23:03:40.684798 19911 webserver.cc:460] Webserver started at http://127.23.194.1:34895/ using document root <none> and password file <none>
I20260808 23:03:40.684990 19911 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:40.685037 19911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:40.685174 19911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:40.686218 19911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "149d9aba9b3842a5b92629a583b8584a"
format_stamp: "Formatted at 2026-08-08 23:03:40 on dist-test-slave-f0tn"
server_key: "e0465cc7875d05af4abd5b6afc24cf0d"
server_key_iv: "4eb5b6b3ab439023198110cdfd8e7d7c"
server_key_version: "encryptionkey@0"
I20260808 23:03:40.686542 19911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "149d9aba9b3842a5b92629a583b8584a"
format_stamp: "Formatted at 2026-08-08 23:03:40 on dist-test-slave-f0tn"
server_key: "e0465cc7875d05af4abd5b6afc24cf0d"
server_key_iv: "4eb5b6b3ab439023198110cdfd8e7d7c"
server_key_version: "encryptionkey@0"
I20260808 23:03:40.687742 19911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:03:40.688373 19933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:40.688520 19911 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:40.688601 19911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "149d9aba9b3842a5b92629a583b8584a"
format_stamp: "Formatted at 2026-08-08 23:03:40 on dist-test-slave-f0tn"
server_key: "e0465cc7875d05af4abd5b6afc24cf0d"
server_key_iv: "4eb5b6b3ab439023198110cdfd8e7d7c"
server_key_version: "encryptionkey@0"
I20260808 23:03:40.688673 19911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:40.699270 19911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:40.701009 19911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:40.701129 19911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:40.701318 19911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:40.701611 19911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:40.701645 19911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:40.701683 19911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:40.701707 19911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:40.707070 19911 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:35955
I20260808 23:03:40.707145 20108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:35955 every 8 connection(s)
I20260808 23:03:40.707389 19911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
Aug 08 23:03:40 dist-test-slave-f0tn krb5kdc[19743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230220, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:03:40.715049 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 19911
I20260808 23:03:40.715173 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:03:40.715389 24328 external_mini_cluster.cc:1428] Setting key ca6c76edad772f8560977140d60ee527
I20260808 23:03:40.715524 19874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:40.708018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:33077 (local address 127.23.194.62:39557)
0808 23:03:40.708067 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:40.708069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:40.708288 (+ 219us) server_negotiation.cc:407] Connection header received
0808 23:03:40.708704 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:40.708705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:40.708716 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:40.708764 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:40.709675 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:40.710189 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:40.711096 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:40.711318 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:40.711373 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:40.713154 (+ 1781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:40.713155 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:40.713156 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:40.713162 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:40.714391 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:40.714702 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:40.714702 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:40.714703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:40.714752 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:40.714940 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:40.714941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:40.714941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:40.715032 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:40.715072 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:40.715375 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 23:03:40.715447 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 23:03:40.715742 20114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:40.707990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:39557 (local address 127.23.194.1:33077)
0808 23:03:40.708146 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:40.708182 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:40.708610 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:40.708814 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:40.708817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:40.709340 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:40.709574 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:40.709579 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:40.710259 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:40.710260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:40.711014 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:40.711016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:40.711378 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:40.711379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:40.711468 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:40.711844 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:03:40.711851 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:40.713077 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:40.714447 (+ 1370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:40.714450 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:40.714452 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:40.714639 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:40.714798 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:40.714799 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:40.714799 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:40.714887 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:40.715080 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:40.715086 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:40.715241 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 23:03:40.715358 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 23:03:40.715512 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260808 23:03:40.716228 20109 heartbeater.cc:347] Connected to a master server at 127.23.194.62:39557
I20260808 23:03:40.716312 20109 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:40.716471 20109 heartbeater.cc:511] Master 127.23.194.62:39557 requested a full tablet report, sending...
I20260808 23:03:40.716966 19816 ts_manager.cc:194] Registered new tserver with Master: 149d9aba9b3842a5b92629a583b8584a (127.23.194.1:35955)
I20260808 23:03:40.717607 19816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:33077
WARNING: no policy specified for kudu/127.23.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
I20260808 23:03:40.738086 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:39557
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:42137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:40.820559 20117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:40.820744 20117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:40.820765 20117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:40.821939 20117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:40.821978 20117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:40.822007 20117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:03:40.823469 20117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:42137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:39557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:40.823702 20117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:40.823899 20117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:40.828508 20122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:40.828547 20127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:40.828672 20125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:40.828732 20117 server_base.cc:1034] running on GCE node
I20260808 23:03:40.828922 20117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:40.829123 20117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:40.830261 20117 hybrid_clock.cc:648] HybridClock initialized: now 1786230220830251 us; error 28 us; skew 500 ppm
Aug 08 23:03:40 dist-test-slave-f0tn krb5kdc[19743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230220, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:40.832508 20117 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 23:03:40.832906 20117 webserver.cc:460] Webserver started at http://127.23.194.2:32879/ using document root <none> and password file <none>
I20260808 23:03:40.833112 20117 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:40.833158 20117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:40.833271 20117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:40.834266 20117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "3404294bf3a44444bc4d493f5bd7a13c"
format_stamp: "Formatted at 2026-08-08 23:03:40 on dist-test-slave-f0tn"
server_key: "ee28ee6211e0a02f3c182feb00436c01"
server_key_iv: "8905bb53892e46defbcea9166d8ad4d0"
server_key_version: "encryptionkey@0"
I20260808 23:03:40.834575 20117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "3404294bf3a44444bc4d493f5bd7a13c"
format_stamp: "Formatted at 2026-08-08 23:03:40 on dist-test-slave-f0tn"
server_key: "ee28ee6211e0a02f3c182feb00436c01"
server_key_iv: "8905bb53892e46defbcea9166d8ad4d0"
server_key_version: "encryptionkey@0"
I20260808 23:03:40.836041 20117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:03:40.836699 20141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:40.836835 20117 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:40.836905 20117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "3404294bf3a44444bc4d493f5bd7a13c"
format_stamp: "Formatted at 2026-08-08 23:03:40 on dist-test-slave-f0tn"
server_key: "ee28ee6211e0a02f3c182feb00436c01"
server_key_iv: "8905bb53892e46defbcea9166d8ad4d0"
server_key_version: "encryptionkey@0"
I20260808 23:03:40.836969 20117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:40.853600 20117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:40.855203 20117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:40.855309 20117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:40.855489 20117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:40.855736 20117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:40.855767 20117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:40.855786 20117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:40.855803 20117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:40.861263 20117 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:43521
I20260808 23:03:40.861325 20303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:43521 every 8 connection(s)
I20260808 23:03:40.861613 20117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:03:40.862304 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 20117
I20260808 23:03:40.862428 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:03:40.862625 24328 external_mini_cluster.cc:1428] Setting key c402c4483bca8a05163205c12a69462b
Aug 08 23:03:40 dist-test-slave-f0tn krb5kdc[19743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230220, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:03:40.870371 19874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:40.862263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:53007 (local address 127.23.194.62:39557)
0808 23:03:40.862307 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:40.862309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:40.862598 (+ 289us) server_negotiation.cc:407] Connection header received
0808 23:03:40.863137 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:40.863138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:40.863150 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:40.863192 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:40.864082 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:40.864564 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:40.865544 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:40.865745 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:40.865800 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:40.867888 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:40.867889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:40.867889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:40.867896 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:40.869118 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:40.869508 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:40.869509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:40.869510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:40.869568 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:40.869780 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:40.869781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:40.869781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:40.869889 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:40.869944 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:40.870201 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 23:03:40.870281 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 23:03:40.870513 20309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:40.862263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:39557 (local address 127.23.194.2:53007)
0808 23:03:40.862467 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:40.862514 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:40.863050 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:40.863263 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:40.863267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:40.863807 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:40.863990 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:40.863994 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:40.864642 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:40.864643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:40.865468 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:40.865472 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:40.865811 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:40.865812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:40.865944 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:40.866390 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:03:40.866399 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:40.867804 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:40.869194 (+ 1390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:40.869200 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:40.869202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:40.869448 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:40.869619 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:40.869620 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:40.869620 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:40.869727 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:40.869942 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:40.869948 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:40.870074 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 23:03:40.870169 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 23:03:40.870313 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":91,"threads_started":1}
I20260808 23:03:40.870935 20304 heartbeater.cc:347] Connected to a master server at 127.23.194.62:39557
I20260808 23:03:40.871021 20304 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:40.871187 20304 heartbeater.cc:511] Master 127.23.194.62:39557 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.23.194.3@KRBTEST.COM; defaulting to no policy
I20260808 23:03:40.871547 19816 ts_manager.cc:194] Registered new tserver with Master: 3404294bf3a44444bc4d493f5bd7a13c (127.23.194.2:43521)
I20260808 23:03:40.871908 19816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:53007
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
I20260808 23:03:40.886212 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:39557
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:42137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:40.967283 20313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:40.967479 20313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:40.967500 20313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:40.968573 20313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:40.968605 20313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:40.968629 20313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:03:40.970083 20313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:42137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:39557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:40.970268 20313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:40.970435 20313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:40.975114 20326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:40.975168 20313 server_base.cc:1034] running on GCE node
W20260808 23:03:40.975126 20321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:40.975096 20322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:40.975452 20313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:40.975682 20313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:40.976812 20313 hybrid_clock.cc:648] HybridClock initialized: now 1786230220976801 us; error 28 us; skew 500 ppm
Aug 08 23:03:40 dist-test-slave-f0tn krb5kdc[19743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230220, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:40.979113 20313 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 23:03:40.979522 20313 webserver.cc:460] Webserver started at http://127.23.194.3:39777/ using document root <none> and password file <none>
I20260808 23:03:40.979734 20313 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:40.979782 20313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:40.979907 20313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:40.980858 20313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "d947b1b500e64e458c4efba3d515335c"
format_stamp: "Formatted at 2026-08-08 23:03:40 on dist-test-slave-f0tn"
server_key: "0ede3fea8ec30882fc41153a282dbba5"
server_key_iv: "b4cc721cc406b999af6549efc0dc496d"
server_key_version: "encryptionkey@0"
I20260808 23:03:40.981173 20313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "d947b1b500e64e458c4efba3d515335c"
format_stamp: "Formatted at 2026-08-08 23:03:40 on dist-test-slave-f0tn"
server_key: "0ede3fea8ec30882fc41153a282dbba5"
server_key_iv: "b4cc721cc406b999af6549efc0dc496d"
server_key_version: "encryptionkey@0"
I20260808 23:03:40.982410 20313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 23:03:40.983039 20338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:40.983186 20313 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:03:40.983268 20313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "d947b1b500e64e458c4efba3d515335c"
format_stamp: "Formatted at 2026-08-08 23:03:40 on dist-test-slave-f0tn"
server_key: "0ede3fea8ec30882fc41153a282dbba5"
server_key_iv: "b4cc721cc406b999af6549efc0dc496d"
server_key_version: "encryptionkey@0"
I20260808 23:03:40.983332 20313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:41.005980 20313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:41.008131 20313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:41.008275 20313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:41.008498 20313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:41.008810 20313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:41.008854 20313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:41.008884 20313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:41.008906 20313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:41.014788 20313 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:43735
I20260808 23:03:41.014850 20508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:43735 every 8 connection(s)
I20260808 23:03:41.015141 20313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
Aug 08 23:03:41 dist-test-slave-f0tn krb5kdc[19743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230220, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:03:41.020677 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 20313
I20260808 23:03:41.020787 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:03:41.020942 24328 external_mini_cluster.cc:1428] Setting key 24f415c0a4e922a8d66b3f100207918f
I20260808 23:03:41.023581 19874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.015814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:43017 (local address 127.23.194.62:39557)
0808 23:03:41.015884 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:41.015887 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:41.016031 (+ 144us) server_negotiation.cc:407] Connection header received
0808 23:03:41.016479 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:41.016480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:41.016488 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:41.016533 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:41.017445 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.017930 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.018836 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.019012 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.019056 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.021065 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:41.021067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:41.021067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:41.021075 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:41.022491 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:41.022833 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:41.022833 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:41.022834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:41.022879 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:41.023096 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:41.023097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:41.023097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:41.023203 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:41.023251 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:41.023461 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 23:03:41.023518 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 23:03:41.023743 20514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.015775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:39557 (local address 127.23.194.3:43017)
0808 23:03:41.015919 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:41.015955 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:41.016408 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:41.016593 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:41.016596 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:41.017145 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:41.017337 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.017342 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.017991 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.017992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.018775 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.018777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.019064 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.019065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.019156 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.019585 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:03:41.019593 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:41.020981 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:41.022555 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:41.022561 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:41.022562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:41.022778 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:41.022941 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:41.022942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:41.022942 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:41.023046 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:41.023249 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:41.023253 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:41.023363 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 23:03:41.023441 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 23:03:41.023556 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260808 23:03:41.024168 20509 heartbeater.cc:347] Connected to a master server at 127.23.194.62:39557
I20260808 23:03:41.024251 20509 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:41.024415 20509 heartbeater.cc:511] Master 127.23.194.62:39557 requested a full tablet report, sending...
I20260808 23:03:41.024752 19816 ts_manager.cc:194] Registered new tserver with Master: d947b1b500e64e458c4efba3d515335c (127.23.194.3:43735)
I20260808 23:03:41.025092 19816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:43017
I20260808 23:03:41.032719 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:03:41.039799 19874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.034044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55852 (local address 127.23.194.62:39557)
0808 23:03:41.034090 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:41.034092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:41.034209 (+ 117us) server_negotiation.cc:407] Connection header received
0808 23:03:41.034266 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:41.034267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:41.034276 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:41.034317 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:41.034819 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.035263 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.035967 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.036166 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.036217 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.037334 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:41.037336 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:41.037336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:41.037344 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:41.038421 (+ 1077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:41.038951 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:41.038952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:41.038952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:41.039016 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:41.039210 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:41.039210 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:41.039211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:41.039336 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:41.039456 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:41.039655 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 23:03:41.039719 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 23:03:41.041108 19816 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:03:41.042670 19816 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:03:41.048520 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.044819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:43521 (local address 127.0.0.1:40552)
0808 23:03:41.045217 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:41.045227 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:41.045268 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:41.045607 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:41.045607 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:41.045610 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:41.045654 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.045656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.046732 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.046733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.047519 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.047522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.048233 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.048234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.048317 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.048331 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 23:03:41.048393 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 23:03:41.048446 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":1,"thread_start_us":72,"threads_started":1}
I20260808 23:03:41.048529 20526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.045020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40552 (local address 127.23.194.2:43521)
0808 23:03:41.045360 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:41.045364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:41.045373 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:03:41.045442 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:41.045444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:41.045460 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:41.045529 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:41.045725 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.046643 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.047613 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.048155 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.048219 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.048279 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:41.048414 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 23:03:41.048454 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":100,"threads_started":1}
I20260808 23:03:41.048833 20521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.044600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:43735 (local address 127.0.0.1:60254)
0808 23:03:41.044727 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:41.044763 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:41.044824 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:41.045163 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:41.045166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:41.045168 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:41.045227 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.045228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.046429 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.046429 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.047654 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.047657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.048566 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.048567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.048629 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.048633 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:03:41.048661 (+ 28us) client_negotiation.cc:240] Negotiation successful
0808 23:03:41.048703 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":33,"thread_start_us":84,"threads_started":1}
I20260808 23:03:41.048936 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.044786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49614 (local address 127.23.194.1:35955)
0808 23:03:41.044990 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:41.044994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:41.045005 (+ 11us) server_negotiation.cc:407] Connection header received
0808 23:03:41.045082 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:41.045084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:41.045100 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:41.045179 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:41.045371 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.046239 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.047756 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.048467 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.048522 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.048578 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:41.048816 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 23:03:41.048861 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":101,"threads_started":1}
I20260808 23:03:41.049063 20522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.044600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:35955 (local address 127.0.0.1:49614)
0808 23:03:41.044936 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:41.044944 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:41.044982 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:41.045272 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:41.045273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:41.045275 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:41.045317 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.045319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.046335 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.046336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.047673 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.047675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.048606 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.048607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.048658 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.048661 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:03:41.048700 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 23:03:41.048730 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":42,"thread_start_us":78,"threads_started":1}
I20260808 23:03:41.049237 20204 tablet_service.cc:1467] Processing CreateTablet for tablet 9ba66b18b8d44fe399db8f27ee4d1b9e (DEFAULT_TABLE table=test-table [id=acacf71a3e5b4f41bfe87d1f2b2001a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:41.049422 20523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.044704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60254 (local address 127.23.194.3:43735)
0808 23:03:41.045016 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:41.045020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:41.045029 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:03:41.045061 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:41.045063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:41.045077 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:41.045160 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:41.045279 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.046329 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.047755 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.048451 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.049235 (+ 784us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.049281 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:41.049301 (+ 20us) server_negotiation.cc:299] Negotiation successful
0808 23:03:41.049331 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":77,"threads_started":1}
I20260808 23:03:41.049609 20204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ba66b18b8d44fe399db8f27ee4d1b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:41.050086 20407 tablet_service.cc:1467] Processing CreateTablet for tablet 9ba66b18b8d44fe399db8f27ee4d1b9e (DEFAULT_TABLE table=test-table [id=acacf71a3e5b4f41bfe87d1f2b2001a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:41.050459 20407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ba66b18b8d44fe399db8f27ee4d1b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:41.052984 20529 tablet_bootstrap.cc:492] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c: Bootstrap starting.
I20260808 23:03:41.053598 20531 tablet_bootstrap.cc:492] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c: Bootstrap starting.
I20260808 23:03:41.053754 20529 tablet_bootstrap.cc:654] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:41.053680 20015 tablet_service.cc:1467] Processing CreateTablet for tablet 9ba66b18b8d44fe399db8f27ee4d1b9e (DEFAULT_TABLE table=test-table [id=acacf71a3e5b4f41bfe87d1f2b2001a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:41.054025 20015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ba66b18b8d44fe399db8f27ee4d1b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:41.054198 20531 tablet_bootstrap.cc:654] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:41.054335 20529 log.cc:824] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:41.054718 20531 log.cc:824] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:41.055234 20529 tablet_bootstrap.cc:492] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c: No bootstrap required, opened a new log
I20260808 23:03:41.055306 20529 ts_tablet_manager.cc:1397] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 23:03:41.055501 20531 tablet_bootstrap.cc:492] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c: No bootstrap required, opened a new log
I20260808 23:03:41.055567 20531 ts_tablet_manager.cc:1397] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:03:41.056740 20534 tablet_bootstrap.cc:492] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a: Bootstrap starting.
I20260808 23:03:41.057180 20529 raft_consensus.cc:348] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "149d9aba9b3842a5b92629a583b8584a" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 35955 } } peers { permanent_uuid: "d947b1b500e64e458c4efba3d515335c" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43735 } } peers { permanent_uuid: "3404294bf3a44444bc4d493f5bd7a13c" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43521 } }
I20260808 23:03:41.057356 20529 raft_consensus.cc:374] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:41.057442 20534 tablet_bootstrap.cc:654] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:41.057444 20529 raft_consensus.cc:729] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3404294bf3a44444bc4d493f5bd7a13c, State: Initialized, Role: FOLLOWER
I20260808 23:03:41.057354 20531 raft_consensus.cc:348] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "149d9aba9b3842a5b92629a583b8584a" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 35955 } } peers { permanent_uuid: "d947b1b500e64e458c4efba3d515335c" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43735 } } peers { permanent_uuid: "3404294bf3a44444bc4d493f5bd7a13c" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43521 } }
I20260808 23:03:41.057541 20531 raft_consensus.cc:374] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:41.057581 20531 raft_consensus.cc:729] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d947b1b500e64e458c4efba3d515335c, State: Initialized, Role: FOLLOWER
I20260808 23:03:41.057582 20529 consensus_queue.cc:260] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "149d9aba9b3842a5b92629a583b8584a" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 35955 } } peers { permanent_uuid: "d947b1b500e64e458c4efba3d515335c" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43735 } } peers { permanent_uuid: "3404294bf3a44444bc4d493f5bd7a13c" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43521 } }
I20260808 23:03:41.057704 20531 consensus_queue.cc:260] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "149d9aba9b3842a5b92629a583b8584a" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 35955 } } peers { permanent_uuid: "d947b1b500e64e458c4efba3d515335c" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43735 } } peers { permanent_uuid: "3404294bf3a44444bc4d493f5bd7a13c" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43521 } }
I20260808 23:03:41.057879 20529 ts_tablet_manager.cc:1428] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 23:03:41.057926 20304 heartbeater.cc:503] Master 127.23.194.62:39557 was elected leader, sending a full tablet report...
I20260808 23:03:41.057943 20531 ts_tablet_manager.cc:1428] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 23:03:41.058039 20509 heartbeater.cc:503] Master 127.23.194.62:39557 was elected leader, sending a full tablet report...
I20260808 23:03:41.058027 20534 log.cc:824] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:41.058665 20534 tablet_bootstrap.cc:492] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a: No bootstrap required, opened a new log
I20260808 23:03:41.058732 20534 ts_tablet_manager.cc:1397] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 23:03:41.060014 20534 raft_consensus.cc:348] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "149d9aba9b3842a5b92629a583b8584a" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 35955 } } peers { permanent_uuid: "d947b1b500e64e458c4efba3d515335c" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43735 } } peers { permanent_uuid: "3404294bf3a44444bc4d493f5bd7a13c" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43521 } }
I20260808 23:03:41.060137 20534 raft_consensus.cc:374] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:41.060156 20534 raft_consensus.cc:729] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 149d9aba9b3842a5b92629a583b8584a, State: Initialized, Role: FOLLOWER
I20260808 23:03:41.060240 20534 consensus_queue.cc:260] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "149d9aba9b3842a5b92629a583b8584a" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 35955 } } peers { permanent_uuid: "d947b1b500e64e458c4efba3d515335c" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43735 } } peers { permanent_uuid: "3404294bf3a44444bc4d493f5bd7a13c" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43521 } }
I20260808 23:03:41.060438 20534 ts_tablet_manager.cc:1428] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 23:03:41.060489 20109 heartbeater.cc:503] Master 127.23.194.62:39557 was elected leader, sending a full tablet report...
W20260808 23:03:41.112133 20308 tablet.cc:2369] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:03:41.124706 20544 raft_consensus.cc:482] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:03:41.124820 20544 raft_consensus.cc:504] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "149d9aba9b3842a5b92629a583b8584a" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 35955 } } peers { permanent_uuid: "d947b1b500e64e458c4efba3d515335c" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43735 } } peers { permanent_uuid: "3404294bf3a44444bc4d493f5bd7a13c" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43521 } }
I20260808 23:03:41.125195 20544 leader_election.cc:288] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d947b1b500e64e458c4efba3d515335c (127.23.194.3:43735), 3404294bf3a44444bc4d493f5bd7a13c (127.23.194.2:43521)
I20260808 23:03:41.127763 20523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.125460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:39469 (local address 127.23.194.3:43735)
0808 23:03:41.125519 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:41.125521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:41.125531 (+ 10us) server_negotiation.cc:407] Connection header received
0808 23:03:41.125559 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:41.125559 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:41.125569 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:41.125629 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:41.125779 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.126247 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.127098 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.127460 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.127519 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.127528 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:41.127665 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 23:03:41.127700 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 23:03:41.127763 20114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.125351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:43735 (local address 127.23.194.1:39469)
0808 23:03:41.125416 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:41.125427 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:41.125471 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:41.125647 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:41.125649 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:41.125651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:41.125719 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.125721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.126312 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.126313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.127018 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.127020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.127526 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.127527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.127597 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.127600 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:03:41.127650 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 23:03:41.127688 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 23:03:41.128023 20526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.125553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:37383 (local address 127.23.194.2:43521)
0808 23:03:41.125618 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:41.125620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:41.125718 (+ 98us) server_negotiation.cc:407] Connection header received
0808 23:03:41.125772 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:41.125773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:41.125782 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:41.125824 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:41.125965 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.126419 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.127299 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.127674 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.127734 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.127742 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:41.127920 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 23:03:41.127959 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 23:03:41.128123 20434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ba66b18b8d44fe399db8f27ee4d1b9e" candidate_uuid: "149d9aba9b3842a5b92629a583b8584a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d947b1b500e64e458c4efba3d515335c" is_pre_election: true
I20260808 23:03:41.128242 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.125501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:43521 (local address 127.23.194.1:37383)
0808 23:03:41.125655 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:41.125664 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:41.125715 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:41.125837 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:41.125838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:41.125839 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:41.125909 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.125911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.126493 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.126494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.127216 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.127218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.127752 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.127753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.127831 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.127835 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:03:41.128114 (+ 279us) client_negotiation.cc:240] Negotiation successful
0808 23:03:41.128163 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":93,"threads_started":1}
I20260808 23:03:41.128309 20434 raft_consensus.cc:2454] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 149d9aba9b3842a5b92629a583b8584a in term 0.
I20260808 23:03:41.128539 19937 leader_election.cc:302] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 149d9aba9b3842a5b92629a583b8584a, d947b1b500e64e458c4efba3d515335c; no voters:
I20260808 23:03:41.128684 20544 raft_consensus.cc:2781] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:03:41.128683 20236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ba66b18b8d44fe399db8f27ee4d1b9e" candidate_uuid: "149d9aba9b3842a5b92629a583b8584a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3404294bf3a44444bc4d493f5bd7a13c" is_pre_election: true
I20260808 23:03:41.128741 20544 raft_consensus.cc:482] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:03:41.128763 20544 raft_consensus.cc:3037] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:41.128808 20236 raft_consensus.cc:2454] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 149d9aba9b3842a5b92629a583b8584a in term 0.
I20260808 23:03:41.129531 20544 raft_consensus.cc:504] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "149d9aba9b3842a5b92629a583b8584a" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 35955 } } peers { permanent_uuid: "d947b1b500e64e458c4efba3d515335c" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43735 } } peers { permanent_uuid: "3404294bf3a44444bc4d493f5bd7a13c" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43521 } }
I20260808 23:03:41.129637 20544 leader_election.cc:288] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a [CANDIDATE]: Term 1 election: Requested vote from peers d947b1b500e64e458c4efba3d515335c (127.23.194.3:43735), 3404294bf3a44444bc4d493f5bd7a13c (127.23.194.2:43521)
I20260808 23:03:41.129799 20434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ba66b18b8d44fe399db8f27ee4d1b9e" candidate_uuid: "149d9aba9b3842a5b92629a583b8584a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d947b1b500e64e458c4efba3d515335c"
I20260808 23:03:41.129814 20236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ba66b18b8d44fe399db8f27ee4d1b9e" candidate_uuid: "149d9aba9b3842a5b92629a583b8584a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3404294bf3a44444bc4d493f5bd7a13c"
I20260808 23:03:41.129879 20236 raft_consensus.cc:3037] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:41.129873 20434 raft_consensus.cc:3037] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:41.130563 20236 raft_consensus.cc:2454] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 149d9aba9b3842a5b92629a583b8584a in term 1.
I20260808 23:03:41.130573 20434 raft_consensus.cc:2454] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 149d9aba9b3842a5b92629a583b8584a in term 1.
I20260808 23:03:41.130714 19937 leader_election.cc:302] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 149d9aba9b3842a5b92629a583b8584a, d947b1b500e64e458c4efba3d515335c; no voters:
I20260808 23:03:41.130815 20544 raft_consensus.cc:2781] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:41.130967 20544 raft_consensus.cc:686] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a [term 1 LEADER]: Becoming Leader. State: Replica: 149d9aba9b3842a5b92629a583b8584a, State: Running, Role: LEADER
I20260808 23:03:41.131047 20544 consensus_queue.cc:237] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "149d9aba9b3842a5b92629a583b8584a" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 35955 } } peers { permanent_uuid: "d947b1b500e64e458c4efba3d515335c" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43735 } } peers { permanent_uuid: "3404294bf3a44444bc4d493f5bd7a13c" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43521 } }
I20260808 23:03:41.131839 19815 catalog_manager.cc:5495] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a reported cstate change: term changed from 0 to 1, leader changed from <none> to 149d9aba9b3842a5b92629a583b8584a (127.23.194.1). New cstate: current_term: 1 leader_uuid: "149d9aba9b3842a5b92629a583b8584a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "149d9aba9b3842a5b92629a583b8584a" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 35955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d947b1b500e64e458c4efba3d515335c" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3404294bf3a44444bc4d493f5bd7a13c" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43521 } health_report { overall_health: UNKNOWN } } }
I20260808 23:03:41.139858 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.137884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49618 (local address 127.23.194.1:35955)
0808 23:03:41.137924 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:41.137926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:41.137935 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:03:41.138009 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:41.138010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:41.138025 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:41.138071 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:03:41.138164 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.138587 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.139241 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.139412 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.139454 (+ 42us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.139557 (+ 103us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:03:41.139659 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:03:41.139748 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:41.139771 (+ 23us) server_negotiation.cc:299] Negotiation successful
0808 23:03:41.139780 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 23:03:41.141582 20434 raft_consensus.cc:1261] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c [term 1 FOLLOWER]: Refusing update from remote peer 149d9aba9b3842a5b92629a583b8584a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:03:41.141585 20236 raft_consensus.cc:1261] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c [term 1 FOLLOWER]: Refusing update from remote peer 149d9aba9b3842a5b92629a583b8584a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:03:41.141896 20546 consensus_queue.cc:1035] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3404294bf3a44444bc4d493f5bd7a13c" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:03:41.142007 20544 consensus_queue.cc:1035] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d947b1b500e64e458c4efba3d515335c" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 43735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:03:41.144205 20549 mvcc.cc:205] Tried to move back new op lower bound from 7316398985793327104 to 7316398985753305088. Current Snapshot: MvccSnapshot[applied={T|T < 7316398985793327104}]
I20260808 23:03:41.144676 20550 mvcc.cc:205] Tried to move back new op lower bound from 7316398985793327104 to 7316398985753305088. Current Snapshot: MvccSnapshot[applied={T|T < 7316398985793327104}]
I20260808 23:03:41.147009 19815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:03:41.147094 19815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:03:41.147771 19815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cf550ee7dd554169b1e7b79318d6d479: Sending DeleteTablet for 3 replicas of tablet 9ba66b18b8d44fe399db8f27ee4d1b9e
I20260808 23:03:41.148061 20015 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ba66b18b8d44fe399db8f27ee4d1b9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:03:41 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:49614
I20260808 23:03:41.148113 20204 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ba66b18b8d44fe399db8f27ee4d1b9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:03:41 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:40552
I20260808 23:03:41.148136 20407 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ba66b18b8d44fe399db8f27ee4d1b9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:03:41 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:60254
I20260808 23:03:41.148280 20565 tablet_replica.cc:330] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a: stopping tablet replica
I20260808 23:03:41.148375 20565 raft_consensus.cc:2229] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 149d9aba9b3842a5b92629a583b8584a [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:03:41.148490 20568 tablet_replica.cc:330] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c: stopping tablet replica
I20260808 23:03:41.148479 20566 tablet_replica.cc:330] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c: stopping tablet replica
I20260808 23:03:41.148583 20568 raft_consensus.cc:2229] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:03:41.148679 20566 raft_consensus.cc:2229] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:03:41.148687 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 19911
I20260808 23:03:41.165666 20568 raft_consensus.cc:2258] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:03:41.168663 20568 ts_tablet_manager.cc:1905] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:03:41.170204 20568 ts_tablet_manager.cc:1918] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:03:41.170300 20568 log.cc:1196] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/9ba66b18b8d44fe399db8f27ee4d1b9e
I20260808 23:03:41.170645 20568 ts_tablet_manager.cc:1939] T 9ba66b18b8d44fe399db8f27ee4d1b9e P d947b1b500e64e458c4efba3d515335c: Deleting consensus metadata
I20260808 23:03:41.171473 19795 catalog_manager.cc:4841] TS d947b1b500e64e458c4efba3d515335c (127.23.194.3:43735): tablet 9ba66b18b8d44fe399db8f27ee4d1b9e (table test-table [id=acacf71a3e5b4f41bfe87d1f2b2001a1]) successfully deleted
I20260808 23:03:41.171566 20566 raft_consensus.cc:2258] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:03:41.172013 20566 ts_tablet_manager.cc:1905] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:03:41.173175 20566 ts_tablet_manager.cc:1918] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:03:41.173242 20566 log.cc:1196] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/9ba66b18b8d44fe399db8f27ee4d1b9e
I20260808 23:03:41.173525 20566 ts_tablet_manager.cc:1939] T 9ba66b18b8d44fe399db8f27ee4d1b9e P 3404294bf3a44444bc4d493f5bd7a13c: Deleting consensus metadata
I20260808 23:03:41.173959 19795 catalog_manager.cc:4841] TS 3404294bf3a44444bc4d493f5bd7a13c (127.23.194.2:43521): tablet 9ba66b18b8d44fe399db8f27ee4d1b9e (table test-table [id=acacf71a3e5b4f41bfe87d1f2b2001a1]) successfully deleted
W20260808 23:03:41.175808 19792 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.194.1:35955 (error 108)
I20260808 23:03:41.176070 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 20117
I20260808 23:03:41.179044 20522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.176158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:35955 (local address 127.0.0.1:49634)
0808 23:03:41.178966 (+ 2808us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:41.179011 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.1:35955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2783}
W20260808 23:03:41.179172 19792 catalog_manager.cc:4584] TS 149d9aba9b3842a5b92629a583b8584a (127.23.194.1:35955): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9ba66b18b8d44fe399db8f27ee4d1b9e: Network error: Client connection negotiation failed: client connection to 127.23.194.1:35955: connect: Connection refused (error 111)
I20260808 23:03:41.187417 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 20313
I20260808 23:03:41.195979 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 19765
2026-08-08T23:03:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2018 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T23:03:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:03:41Z Disabled control of system clock
I20260808 23:03:41.226949 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:41887
--webserver_interface=127.23.194.62
--webserver_port=0
--builtin_ntp_servers=127.23.194.20:36627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:41887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:03:41.306572 20577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:41.306784 20577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:41.306803 20577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:41.307871 20577 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:03:41.307902 20577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:41.307914 20577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:41.307924 20577 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:03:41.307934 20577 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:03:41.309218 20577 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:36627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:41887
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:41887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:41.309449 20577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:41.309693 20577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:41.314211 20584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:41.314239 20588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:41.314352 20577 server_base.cc:1034] running on GCE node
W20260808 23:03:41.314220 20586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:41.314730 20577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:41.315007 20577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:41.316145 20577 hybrid_clock.cc:648] HybridClock initialized: now 1786230221316124 us; error 32 us; skew 500 ppm
I20260808 23:03:41.317312 20577 webserver.cc:460] Webserver started at http://127.23.194.62:38013/ using document root <none> and password file <none>
I20260808 23:03:41.317551 20577 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:41.317598 20577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:41.317723 20577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:41.318753 20577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "db7d1683607a47179c7257c4249824bd"
format_stamp: "Formatted at 2026-08-08 23:03:41 on dist-test-slave-f0tn"
server_key: "5dbf90847240ee7dea3346c38fdf76a3"
server_key_iv: "a9e00b73384a41be2e784bffad9b43be"
server_key_version: "encryptionkey@0"
I20260808 23:03:41.319072 20577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "db7d1683607a47179c7257c4249824bd"
format_stamp: "Formatted at 2026-08-08 23:03:41 on dist-test-slave-f0tn"
server_key: "5dbf90847240ee7dea3346c38fdf76a3"
server_key_iv: "a9e00b73384a41be2e784bffad9b43be"
server_key_version: "encryptionkey@0"
I20260808 23:03:41.320292 20577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:03:41.321010 20598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:41.321165 20577 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:41.321247 20577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "db7d1683607a47179c7257c4249824bd"
format_stamp: "Formatted at 2026-08-08 23:03:41 on dist-test-slave-f0tn"
server_key: "5dbf90847240ee7dea3346c38fdf76a3"
server_key_iv: "a9e00b73384a41be2e784bffad9b43be"
server_key_version: "encryptionkey@0"
I20260808 23:03:41.321311 20577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:41.330050 20577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:41.330325 20577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:41.330444 20577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:41.334298 20577 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:41887
I20260808 23:03:41.334333 20670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:41887 every 8 connection(s)
I20260808 23:03:41.334635 20577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:03:41.335624 20671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:41.338011 20671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd: Bootstrap starting.
I20260808 23:03:41.338572 20671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:41.338817 20671 log.cc:824] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:41.339413 20671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd: No bootstrap required, opened a new log
I20260808 23:03:41.340898 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 20577
I20260808 23:03:41.340999 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:03:41.340963 20671 raft_consensus.cc:348] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7d1683607a47179c7257c4249824bd" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 41887 } }
I20260808 23:03:41.341126 20671 raft_consensus.cc:374] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:41.341154 20671 raft_consensus.cc:729] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db7d1683607a47179c7257c4249824bd, State: Initialized, Role: FOLLOWER
I20260808 23:03:41.341169 24328 external_mini_cluster.cc:1428] Setting key 7795baae586ac457c0196ce9a5f55c89
I20260808 23:03:41.341293 20671 consensus_queue.cc:260] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7d1683607a47179c7257c4249824bd" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 41887 } }
I20260808 23:03:41.341369 20671 raft_consensus.cc:388] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:03:41.341437 20671 raft_consensus.cc:482] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:03:41.341495 20671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:41.342180 20671 raft_consensus.cc:504] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7d1683607a47179c7257c4249824bd" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 41887 } }
I20260808 23:03:41.342288 20671 leader_election.cc:302] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db7d1683607a47179c7257c4249824bd; no voters:
I20260808 23:03:41.342442 20671 leader_election.cc:288] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:03:41.342483 20678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:41.342669 20678 raft_consensus.cc:686] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd [term 1 LEADER]: Becoming Leader. State: Replica: db7d1683607a47179c7257c4249824bd, State: Running, Role: LEADER
I20260808 23:03:41.342777 20671 sys_catalog.cc:564] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:03:41.342763 20678 consensus_queue.cc:237] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7d1683607a47179c7257c4249824bd" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 41887 } }
I20260808 23:03:41.343070 20678 sys_catalog.cc:455] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db7d1683607a47179c7257c4249824bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7d1683607a47179c7257c4249824bd" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 41887 } } }
I20260808 23:03:41.343204 20678 sys_catalog.cc:458] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:41.343464 20678 sys_catalog.cc:455] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader db7d1683607a47179c7257c4249824bd. Latest consensus state: current_term: 1 leader_uuid: "db7d1683607a47179c7257c4249824bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7d1683607a47179c7257c4249824bd" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 41887 } } }
I20260808 23:03:41.343528 20678 sys_catalog.cc:458] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:41.343856 20689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:03:41.344369 20689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:03:41.345834 20689 catalog_manager.cc:1347] Generated new cluster ID: 44f103cd49934b6a9b64709ce07f2d0a
I20260808 23:03:41.345881 20689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:03:41.347574 20677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.341805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59548 (local address 127.23.194.62:41887)
0808 23:03:41.342012 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:41.342030 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:41.342064 (+ 34us) server_negotiation.cc:407] Connection header received
0808 23:03:41.342175 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:41.342183 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:41.342273 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:41.342455 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:41.342666 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.344370 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.345509 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.346085 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.346206 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.346859 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:41.346863 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:41.346863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:41.347090 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:41.347134 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:41.347150 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:41.347215 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:41.347283 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 23:03:41.347363 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
I20260808 23:03:41.354432 20689 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:03:41.355127 20689 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:03:41.362176 20689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd: Generated new TSK 0
I20260808 23:03:41.362339 20689 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:03:41.367755 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:41887
--builtin_ntp_servers=127.23.194.20:36627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:03:41.448278 20706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:41.448460 20706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:41.448480 20706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:41.449606 20706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:41.449648 20706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:41.449671 20706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:03:41.451057 20706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:36627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:41887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:41.451273 20706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:41.451547 20706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:41.456437 20712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:41.456481 20711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:41.456578 20706 server_base.cc:1034] running on GCE node
W20260808 23:03:41.456650 20714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:41.456914 20706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:41.457188 20706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:41.458341 20706 hybrid_clock.cc:648] HybridClock initialized: now 1786230221458327 us; error 27 us; skew 500 ppm
I20260808 23:03:41.459769 20706 webserver.cc:460] Webserver started at http://127.23.194.1:33093/ using document root <none> and password file <none>
I20260808 23:03:41.460058 20706 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:41.460126 20706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:41.460279 20706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:41.461737 20706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "c78388c05a134d6cad559178c7e0dd76"
format_stamp: "Formatted at 2026-08-08 23:03:41 on dist-test-slave-f0tn"
server_key: "affae75911ddd29eec4cf91fb7c95149"
server_key_iv: "f9b452d5d2e608b80627b098da740f75"
server_key_version: "encryptionkey@0"
I20260808 23:03:41.462141 20706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "c78388c05a134d6cad559178c7e0dd76"
format_stamp: "Formatted at 2026-08-08 23:03:41 on dist-test-slave-f0tn"
server_key: "affae75911ddd29eec4cf91fb7c95149"
server_key_iv: "f9b452d5d2e608b80627b098da740f75"
server_key_version: "encryptionkey@0"
I20260808 23:03:41.463867 20706 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 23:03:41.464733 20724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:41.464896 20706 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:41.464998 20706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "c78388c05a134d6cad559178c7e0dd76"
format_stamp: "Formatted at 2026-08-08 23:03:41 on dist-test-slave-f0tn"
server_key: "affae75911ddd29eec4cf91fb7c95149"
server_key_iv: "f9b452d5d2e608b80627b098da740f75"
server_key_version: "encryptionkey@0"
I20260808 23:03:41.465071 20706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:41.482784 20706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:41.483037 20706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:41.483141 20706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:41.483327 20706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:41.483609 20706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:41.483642 20706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:41.483670 20706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:41.483691 20706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:41.489528 20706 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:41123
I20260808 23:03:41.489567 20907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:41123 every 8 connection(s)
I20260808 23:03:41.489889 20706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:03:41.493008 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 20706
I20260808 23:03:41.493122 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:03:41.493330 24328 external_mini_cluster.cc:1428] Setting key 85d0cd733bf7f8b4c666d3359de37b63
I20260808 23:03:41.494266 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:41887
--builtin_ntp_servers=127.23.194.20:36627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 23:03:41.495257 20677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.490532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:38945 (local address 127.23.194.62:41887)
0808 23:03:41.490617 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:41.490620 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:41.490871 (+ 251us) server_negotiation.cc:407] Connection header received
0808 23:03:41.491349 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:41.491350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:41.491363 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:41.491422 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:41.492043 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.492539 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.493815 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.494096 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.494160 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.495025 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:41.495027 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:41.495027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:41.495031 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:41.495046 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:41.495050 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:41.495099 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:41.495175 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 23:03:41.495199 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 23:03:41.495472 20915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.490532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:41887 (local address 127.23.194.1:38945)
0808 23:03:41.490727 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:41.490764 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:41.491236 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:41.491479 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:41.491482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:41.491490 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:41.491963 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.491967 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.492599 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.492600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.493732 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.493735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.494188 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.494189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.494342 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.494852 (+ 510us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:03:41.494853 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:41.494890 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:03:41.494916 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:41.495108 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:41.495112 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:41.495115 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:03:41.495175 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 23:03:41.495273 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":99,"threads_started":1}
I20260808 23:03:41.495959 20912 heartbeater.cc:347] Connected to a master server at 127.23.194.62:41887
I20260808 23:03:41.496045 20912 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:41.496208 20912 heartbeater.cc:511] Master 127.23.194.62:41887 requested a full tablet report, sending...
I20260808 23:03:41.496708 20620 ts_manager.cc:194] Registered new tserver with Master: c78388c05a134d6cad559178c7e0dd76 (127.23.194.1:41123)
I20260808 23:03:41.497556 20620 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.194.1:38945
W20260808 23:03:41.574445 20918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:41.574631 20918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:41.574661 20918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:41.575778 20918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:41.575816 20918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:41.575848 20918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:03:41.577165 20918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:36627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:41887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:41.577415 20918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:41.577690 20918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:41.582410 20929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:41.582490 20933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:41.582472 20918 server_base.cc:1034] running on GCE node
W20260808 23:03:41.582419 20931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:41.582890 20918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:41.583096 20918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:41.584235 20918 hybrid_clock.cc:648] HybridClock initialized: now 1786230221584225 us; error 28 us; skew 500 ppm
I20260808 23:03:41.585356 20918 webserver.cc:460] Webserver started at http://127.23.194.2:46861/ using document root <none> and password file <none>
I20260808 23:03:41.585604 20918 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:41.585652 20918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:41.585768 20918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:41.586716 20918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "be6196364a8a4cfda75b960699f8e7a8"
format_stamp: "Formatted at 2026-08-08 23:03:41 on dist-test-slave-f0tn"
server_key: "014bed032333e38607385783590bc2f3"
server_key_iv: "60337bf6b5c937129ee6ca4f6666aa75"
server_key_version: "encryptionkey@0"
I20260808 23:03:41.587029 20918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "be6196364a8a4cfda75b960699f8e7a8"
format_stamp: "Formatted at 2026-08-08 23:03:41 on dist-test-slave-f0tn"
server_key: "014bed032333e38607385783590bc2f3"
server_key_iv: "60337bf6b5c937129ee6ca4f6666aa75"
server_key_version: "encryptionkey@0"
I20260808 23:03:41.588181 20918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:03:41.588769 20945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:41.588917 20918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:41.588987 20918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "be6196364a8a4cfda75b960699f8e7a8"
format_stamp: "Formatted at 2026-08-08 23:03:41 on dist-test-slave-f0tn"
server_key: "014bed032333e38607385783590bc2f3"
server_key_iv: "60337bf6b5c937129ee6ca4f6666aa75"
server_key_version: "encryptionkey@0"
I20260808 23:03:41.589044 20918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:41.607345 20918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:41.607604 20918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:41.607731 20918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:41.607933 20918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:41.608239 20918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:41.608274 20918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:41.608304 20918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:41.608320 20918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:41.613394 20918 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:35687
I20260808 23:03:41.613457 21123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:35687 every 8 connection(s)
I20260808 23:03:41.613734 20918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:03:41.618459 20677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.614388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:38619 (local address 127.23.194.62:41887)
0808 23:03:41.614452 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:41.614454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:41.614665 (+ 211us) server_negotiation.cc:407] Connection header received
0808 23:03:41.615107 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:41.615109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:41.615118 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:41.615181 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:41.615768 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.616250 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.617113 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.617296 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.617337 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.618132 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:41.618134 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:41.618134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:41.618139 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:41.618165 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:41.618170 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:41.618223 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:41.618323 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 23:03:41.618375 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 23:03:41.618691 21132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.614348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:41887 (local address 127.23.194.2:38619)
0808 23:03:41.614541 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:41.614579 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:41.615032 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:41.615236 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:41.615240 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:41.615247 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:41.615707 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.615710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.616307 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.616307 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.617056 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.617058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.617347 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.617348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.617492 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.617984 (+ 492us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:03:41.617985 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:41.618024 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:03:41.618051 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:41.618230 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:41.618238 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:41.618242 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:03:41.618353 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 23:03:41.618458 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260808 23:03:41.619379 21124 heartbeater.cc:347] Connected to a master server at 127.23.194.62:41887
I20260808 23:03:41.619473 21124 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:41.619633 21124 heartbeater.cc:511] Master 127.23.194.62:41887 requested a full tablet report, sending...
I20260808 23:03:41.619879 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 20918
I20260808 23:03:41.619999 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:03:41.620071 20620 ts_manager.cc:194] Registered new tserver with Master: be6196364a8a4cfda75b960699f8e7a8 (127.23.194.2:35687)
I20260808 23:03:41.620239 24328 external_mini_cluster.cc:1428] Setting key 2b61c7290919c9ac2d127da97321e8d9
I20260808 23:03:41.620534 20620 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.194.2:38619
I20260808 23:03:41.621331 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:41887
--builtin_ntp_servers=127.23.194.20:36627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:03:41.703567 21136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:41.703753 21136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:41.703773 21136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:41.704852 21136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:41.704890 21136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:41.704912 21136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:03:41.706241 21136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:36627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:41887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:41.706431 21136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:41.706661 21136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:41.711313 21141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:41.711375 21136 server_base.cc:1034] running on GCE node
W20260808 23:03:41.711357 21144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:41.711314 21151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:41.711735 21136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:41.711974 21136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:41.713124 21136 hybrid_clock.cc:648] HybridClock initialized: now 1786230221713111 us; error 24 us; skew 500 ppm
I20260808 23:03:41.714294 21136 webserver.cc:460] Webserver started at http://127.23.194.3:35067/ using document root <none> and password file <none>
I20260808 23:03:41.714505 21136 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:41.714553 21136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:41.714679 21136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:41.715662 21136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "7b2f91380e4c48daa18988d72ab686fa"
format_stamp: "Formatted at 2026-08-08 23:03:41 on dist-test-slave-f0tn"
server_key: "e5e69c5340dc7f6b3146c7a47ba1e8d4"
server_key_iv: "21577235472d2fbb287e574c614ba40a"
server_key_version: "encryptionkey@0"
I20260808 23:03:41.716001 21136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "7b2f91380e4c48daa18988d72ab686fa"
format_stamp: "Formatted at 2026-08-08 23:03:41 on dist-test-slave-f0tn"
server_key: "e5e69c5340dc7f6b3146c7a47ba1e8d4"
server_key_iv: "21577235472d2fbb287e574c614ba40a"
server_key_version: "encryptionkey@0"
I20260808 23:03:41.717147 21136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:03:41.717859 21159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:41.718014 21136 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:41.718103 21136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "7b2f91380e4c48daa18988d72ab686fa"
format_stamp: "Formatted at 2026-08-08 23:03:41 on dist-test-slave-f0tn"
server_key: "e5e69c5340dc7f6b3146c7a47ba1e8d4"
server_key_iv: "21577235472d2fbb287e574c614ba40a"
server_key_version: "encryptionkey@0"
I20260808 23:03:41.718169 21136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:41.724617 21136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:41.724841 21136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:41.724951 21136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:41.725142 21136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:41.725463 21136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:41.725504 21136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:41.725530 21136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:41.725554 21136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:41.731053 21136 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:38589
I20260808 23:03:41.731086 21329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:38589 every 8 connection(s)
I20260808 23:03:41.731369 21136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 23:03:41.735751 20677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.732024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:49161 (local address 127.23.194.62:41887)
0808 23:03:41.732083 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:41.732085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:41.732267 (+ 182us) server_negotiation.cc:407] Connection header received
0808 23:03:41.732681 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:41.732681 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:41.732691 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:41.732739 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:41.733313 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.733836 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.734734 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.734909 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.734948 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.735564 (+ 616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:41.735565 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:41.735566 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:41.735569 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:41.735582 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:41.735585 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:41.735621 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:41.735671 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 23:03:41.735695 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 23:03:41.735913 21335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.731990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:41887 (local address 127.23.194.3:49161)
0808 23:03:41.732157 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:41.732193 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:41.732610 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:41.732793 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:41.732796 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:41.732804 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:41.733258 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.733262 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.733897 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.733897 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.734668 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.734670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.734956 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.734957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.735076 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.735456 (+ 380us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:03:41.735456 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:41.735480 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:03:41.735499 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:41.735622 (+ 123us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:41.735624 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:41.735626 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 23:03:41.735668 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 23:03:41.735734 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
I20260808 23:03:41.736373 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 21136
I20260808 23:03:41.736385 21333 heartbeater.cc:347] Connected to a master server at 127.23.194.62:41887
I20260808 23:03:41.736462 21333 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:41.736478 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:03:41.736670 24328 external_mini_cluster.cc:1428] Setting key cfccb6796af655411b6ced8e518bc2fe
I20260808 23:03:41.736696 21333 heartbeater.cc:511] Master 127.23.194.62:41887 requested a full tablet report, sending...
I20260808 23:03:41.737054 20620 ts_manager.cc:194] Registered new tserver with Master: 7b2f91380e4c48daa18988d72ab686fa (127.23.194.3:38589)
I20260808 23:03:41.737485 20620 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.194.3:49161
I20260808 23:03:41.737560 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:03:41.741457 20677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.738826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59562 (local address 127.23.194.62:41887)
0808 23:03:41.738877 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:41.738879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:41.738980 (+ 101us) server_negotiation.cc:407] Connection header received
0808 23:03:41.739046 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:41.739047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:41.739057 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:41.739108 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:41.739254 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.739704 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.740407 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.740606 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.740656 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.741177 (+ 521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:41.741178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:41.741178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:41.741182 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:41.741195 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:41.741198 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:41.741273 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:41.741343 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 23:03:41.741352 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 23:03:41.742657 20620 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:03:41.744247 20620 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:03:41.749949 21344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.746122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:35687 (local address 127.0.0.1:39578)
0808 23:03:41.746292 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:41.746328 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:41.746390 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:41.746794 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:41.746795 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:41.746797 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:41.746839 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.746841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.747766 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.747767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.748898 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.748901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.749676 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.749677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.749751 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.749763 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 23:03:41.749831 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 23:03:41.749878 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":111,"threads_started":1}
I20260808 23:03:41.749959 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.746299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39578 (local address 127.23.194.2:35687)
0808 23:03:41.746661 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:41.746665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:41.746674 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:03:41.746709 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:41.746711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:41.746724 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:41.746793 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:41.746893 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.747690 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.748992 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.749597 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.749665 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.749753 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:41.749845 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 23:03:41.749880 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":104,"threads_started":1}
I20260808 23:03:41.750114 21348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.746331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:41123 (local address 127.0.0.1:44806)
0808 23:03:41.746541 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:41.746550 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:41.746601 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:41.746950 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:41.746951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:41.746954 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:41.746987 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.746988 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.748158 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.748159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.748879 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.748881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.749890 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.749891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.749948 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.749953 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:03:41.749996 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 23:03:41.750037 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":70,"threads_started":1}
I20260808 23:03:41.750347 21351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.746523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44806 (local address 127.23.194.1:41123)
0808 23:03:41.746748 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:41.746753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:41.746762 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:03:41.746799 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:41.746801 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:41.746815 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:41.746885 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:41.747060 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.748097 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.748973 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.749709 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.749774 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.749853 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:41.750229 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 23:03:41.750268 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":94,"threads_started":1}
I20260808 23:03:41.750581 21343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.746135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:38589 (local address 127.0.0.1:58528)
0808 23:03:41.746489 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:41.746497 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:41.746549 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:41.746720 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:41.746723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:41.746725 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:41.746785 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.746787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.748107 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.748108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.749142 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.749144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.750123 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.750124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.750344 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.750348 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:03:41.750482 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 23:03:41.750520 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":9,"thread_start_us":60,"threads_started":1}
I20260808 23:03:41.750798 21345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.746222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58528 (local address 127.23.194.3:38589)
0808 23:03:41.746401 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:41.746405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:41.746568 (+ 163us) server_negotiation.cc:407] Connection header received
0808 23:03:41.746610 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:41.746613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:41.746627 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:41.746711 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:41.746868 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.748017 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.749231 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.749920 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.749977 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.750305 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:41.750689 (+ 384us) server_negotiation.cc:299] Negotiation successful
0808 23:03:41.750726 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":70,"threads_started":1}
I20260808 23:03:41.750764 21023 tablet_service.cc:1467] Processing CreateTablet for tablet d0aa9bd08bbc48dd9f017c50582a75cd (DEFAULT_TABLE table=test-table [id=a3bcd0d1bc9146af92b9173bbfb9d731]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:41.750967 20807 tablet_service.cc:1467] Processing CreateTablet for tablet d0aa9bd08bbc48dd9f017c50582a75cd (DEFAULT_TABLE table=test-table [id=a3bcd0d1bc9146af92b9173bbfb9d731]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:41.751133 21023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0aa9bd08bbc48dd9f017c50582a75cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:41.751283 20807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0aa9bd08bbc48dd9f017c50582a75cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:41.754032 21229 tablet_service.cc:1467] Processing CreateTablet for tablet d0aa9bd08bbc48dd9f017c50582a75cd (DEFAULT_TABLE table=test-table [id=a3bcd0d1bc9146af92b9173bbfb9d731]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:41.754379 21229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0aa9bd08bbc48dd9f017c50582a75cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:41.754400 21352 tablet_bootstrap.cc:492] T d0aa9bd08bbc48dd9f017c50582a75cd P c78388c05a134d6cad559178c7e0dd76: Bootstrap starting.
I20260808 23:03:41.754945 21352 tablet_bootstrap.cc:654] T d0aa9bd08bbc48dd9f017c50582a75cd P c78388c05a134d6cad559178c7e0dd76: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:41.755226 21352 log.cc:824] T d0aa9bd08bbc48dd9f017c50582a75cd P c78388c05a134d6cad559178c7e0dd76: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:41.755896 21352 tablet_bootstrap.cc:492] T d0aa9bd08bbc48dd9f017c50582a75cd P c78388c05a134d6cad559178c7e0dd76: No bootstrap required, opened a new log
I20260808 23:03:41.755956 21352 ts_tablet_manager.cc:1397] T d0aa9bd08bbc48dd9f017c50582a75cd P c78388c05a134d6cad559178c7e0dd76: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:03:41.757099 21358 tablet_bootstrap.cc:492] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa: Bootstrap starting.
I20260808 23:03:41.757805 21358 tablet_bootstrap.cc:654] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:41.758082 21362 tablet_bootstrap.cc:492] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8: Bootstrap starting.
I20260808 23:03:41.757994 21352 raft_consensus.cc:348] T d0aa9bd08bbc48dd9f017c50582a75cd P c78388c05a134d6cad559178c7e0dd76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6196364a8a4cfda75b960699f8e7a8" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 35687 } } peers { permanent_uuid: "c78388c05a134d6cad559178c7e0dd76" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41123 } } peers { permanent_uuid: "7b2f91380e4c48daa18988d72ab686fa" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38589 } }
I20260808 23:03:41.758157 21352 raft_consensus.cc:374] T d0aa9bd08bbc48dd9f017c50582a75cd P c78388c05a134d6cad559178c7e0dd76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:41.758196 21352 raft_consensus.cc:729] T d0aa9bd08bbc48dd9f017c50582a75cd P c78388c05a134d6cad559178c7e0dd76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78388c05a134d6cad559178c7e0dd76, State: Initialized, Role: FOLLOWER
I20260808 23:03:41.758327 21352 consensus_queue.cc:260] T d0aa9bd08bbc48dd9f017c50582a75cd P c78388c05a134d6cad559178c7e0dd76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6196364a8a4cfda75b960699f8e7a8" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 35687 } } peers { permanent_uuid: "c78388c05a134d6cad559178c7e0dd76" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41123 } } peers { permanent_uuid: "7b2f91380e4c48daa18988d72ab686fa" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38589 } }
I20260808 23:03:41.758598 21352 ts_tablet_manager.cc:1428] T d0aa9bd08bbc48dd9f017c50582a75cd P c78388c05a134d6cad559178c7e0dd76: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 23:03:41.758698 21362 tablet_bootstrap.cc:654] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:41.758715 20912 heartbeater.cc:503] Master 127.23.194.62:41887 was elected leader, sending a full tablet report...
I20260808 23:03:41.759006 21362 log.cc:824] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:41.759521 21358 log.cc:824] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:41.760560 21358 tablet_bootstrap.cc:492] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa: No bootstrap required, opened a new log
I20260808 23:03:41.760637 21358 ts_tablet_manager.cc:1397] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260808 23:03:41.760900 21362 tablet_bootstrap.cc:492] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8: No bootstrap required, opened a new log
I20260808 23:03:41.760963 21362 ts_tablet_manager.cc:1397] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 23:03:41.763139 21358 raft_consensus.cc:348] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6196364a8a4cfda75b960699f8e7a8" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 35687 } } peers { permanent_uuid: "c78388c05a134d6cad559178c7e0dd76" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41123 } } peers { permanent_uuid: "7b2f91380e4c48daa18988d72ab686fa" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38589 } }
I20260808 23:03:41.763309 21358 raft_consensus.cc:374] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:41.763352 21358 raft_consensus.cc:729] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b2f91380e4c48daa18988d72ab686fa, State: Initialized, Role: FOLLOWER
I20260808 23:03:41.763398 21362 raft_consensus.cc:348] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6196364a8a4cfda75b960699f8e7a8" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 35687 } } peers { permanent_uuid: "c78388c05a134d6cad559178c7e0dd76" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41123 } } peers { permanent_uuid: "7b2f91380e4c48daa18988d72ab686fa" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38589 } }
I20260808 23:03:41.763474 21358 consensus_queue.cc:260] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6196364a8a4cfda75b960699f8e7a8" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 35687 } } peers { permanent_uuid: "c78388c05a134d6cad559178c7e0dd76" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41123 } } peers { permanent_uuid: "7b2f91380e4c48daa18988d72ab686fa" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38589 } }
I20260808 23:03:41.763540 21362 raft_consensus.cc:374] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:41.763581 21362 raft_consensus.cc:729] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be6196364a8a4cfda75b960699f8e7a8, State: Initialized, Role: FOLLOWER
I20260808 23:03:41.763691 21362 consensus_queue.cc:260] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6196364a8a4cfda75b960699f8e7a8" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 35687 } } peers { permanent_uuid: "c78388c05a134d6cad559178c7e0dd76" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41123 } } peers { permanent_uuid: "7b2f91380e4c48daa18988d72ab686fa" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38589 } }
I20260808 23:03:41.763739 21358 ts_tablet_manager.cc:1428] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 23:03:41.763770 21333 heartbeater.cc:503] Master 127.23.194.62:41887 was elected leader, sending a full tablet report...
I20260808 23:03:41.763926 21362 ts_tablet_manager.cc:1428] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 23:03:41.764005 21124 heartbeater.cc:503] Master 127.23.194.62:41887 was elected leader, sending a full tablet report...
W20260808 23:03:41.864248 21125 tablet.cc:2369] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:03:41.964644 21368 raft_consensus.cc:482] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:03:41.964828 21368 raft_consensus.cc:504] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6196364a8a4cfda75b960699f8e7a8" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 35687 } } peers { permanent_uuid: "c78388c05a134d6cad559178c7e0dd76" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41123 } } peers { permanent_uuid: "7b2f91380e4c48daa18988d72ab686fa" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38589 } }
I20260808 23:03:41.965240 21368 leader_election.cc:288] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78388c05a134d6cad559178c7e0dd76 (127.23.194.1:41123), 7b2f91380e4c48daa18988d72ab686fa (127.23.194.3:38589)
I20260808 23:03:41.968981 21345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.965558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:51275 (local address 127.23.194.3:38589)
0808 23:03:41.965604 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:41.965606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:41.965733 (+ 127us) server_negotiation.cc:407] Connection header received
0808 23:03:41.965769 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:41.965769 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:41.965778 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:41.965830 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:41.965954 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.966678 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.968101 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.968650 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.968723 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.968734 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:41.968865 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 23:03:41.968912 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 23:03:41.969000 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.965488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:38589 (local address 127.23.194.2:51275)
0808 23:03:41.965667 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:41.965675 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:41.965725 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:41.965837 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:41.965837 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:41.965838 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:41.965899 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.965901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.966756 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.966757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.967980 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.967981 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.968725 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.968726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.968800 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.968803 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:03:41.968845 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 23:03:41.968876 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":58,"thread_start_us":97,"threads_started":1}
I20260808 23:03:41.969079 21351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.965589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:46475 (local address 127.23.194.1:41123)
0808 23:03:41.965629 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:41.965631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:41.965640 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:03:41.965674 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:41.965675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:41.965684 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:41.965756 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:41.965908 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.966672 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.968085 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.968650 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.968728 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.968735 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:41.968942 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 23:03:41.968987 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 23:03:41.969084 21132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:41.965488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:41123 (local address 127.23.194.2:46475)
0808 23:03:41.965569 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:41.965580 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:41.965627 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:41.965768 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:41.965769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:41.965771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:41.965840 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.965841 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.966756 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.966757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.967976 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:41.967981 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:41.968773 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:41.968775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:41.968850 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:41.968853 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:03:41.968922 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 23:03:41.968966 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":3}
I20260808 23:03:41.969471 20832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0aa9bd08bbc48dd9f017c50582a75cd" candidate_uuid: "be6196364a8a4cfda75b960699f8e7a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78388c05a134d6cad559178c7e0dd76" is_pre_election: true
I20260808 23:03:41.969487 21255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0aa9bd08bbc48dd9f017c50582a75cd" candidate_uuid: "be6196364a8a4cfda75b960699f8e7a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b2f91380e4c48daa18988d72ab686fa" is_pre_election: true
I20260808 23:03:41.969636 20832 raft_consensus.cc:2454] T d0aa9bd08bbc48dd9f017c50582a75cd P c78388c05a134d6cad559178c7e0dd76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be6196364a8a4cfda75b960699f8e7a8 in term 0.
I20260808 23:03:41.969658 21255 raft_consensus.cc:2454] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be6196364a8a4cfda75b960699f8e7a8 in term 0.
I20260808 23:03:41.969864 20950 leader_election.cc:302] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b2f91380e4c48daa18988d72ab686fa, be6196364a8a4cfda75b960699f8e7a8; no voters:
I20260808 23:03:41.970055 21368 raft_consensus.cc:2781] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:03:41.970124 21368 raft_consensus.cc:482] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:03:41.970140 21368 raft_consensus.cc:3037] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:41.970839 21368 raft_consensus.cc:504] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6196364a8a4cfda75b960699f8e7a8" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 35687 } } peers { permanent_uuid: "c78388c05a134d6cad559178c7e0dd76" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41123 } } peers { permanent_uuid: "7b2f91380e4c48daa18988d72ab686fa" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38589 } }
I20260808 23:03:41.970957 21368 leader_election.cc:288] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [CANDIDATE]: Term 1 election: Requested vote from peers c78388c05a134d6cad559178c7e0dd76 (127.23.194.1:41123), 7b2f91380e4c48daa18988d72ab686fa (127.23.194.3:38589)
I20260808 23:03:41.971123 21255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0aa9bd08bbc48dd9f017c50582a75cd" candidate_uuid: "be6196364a8a4cfda75b960699f8e7a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b2f91380e4c48daa18988d72ab686fa"
I20260808 23:03:41.971166 20832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0aa9bd08bbc48dd9f017c50582a75cd" candidate_uuid: "be6196364a8a4cfda75b960699f8e7a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78388c05a134d6cad559178c7e0dd76"
I20260808 23:03:41.971212 21255 raft_consensus.cc:3037] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:41.971236 20832 raft_consensus.cc:3037] T d0aa9bd08bbc48dd9f017c50582a75cd P c78388c05a134d6cad559178c7e0dd76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:41.972126 20832 raft_consensus.cc:2454] T d0aa9bd08bbc48dd9f017c50582a75cd P c78388c05a134d6cad559178c7e0dd76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6196364a8a4cfda75b960699f8e7a8 in term 1.
I20260808 23:03:41.972126 21255 raft_consensus.cc:2454] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6196364a8a4cfda75b960699f8e7a8 in term 1.
I20260808 23:03:41.972303 20946 leader_election.cc:302] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be6196364a8a4cfda75b960699f8e7a8, c78388c05a134d6cad559178c7e0dd76; no voters:
I20260808 23:03:41.972397 21368 raft_consensus.cc:2781] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:41.972553 21368 raft_consensus.cc:686] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [term 1 LEADER]: Becoming Leader. State: Replica: be6196364a8a4cfda75b960699f8e7a8, State: Running, Role: LEADER
I20260808 23:03:41.972640 21368 consensus_queue.cc:237] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6196364a8a4cfda75b960699f8e7a8" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 35687 } } peers { permanent_uuid: "c78388c05a134d6cad559178c7e0dd76" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41123 } } peers { permanent_uuid: "7b2f91380e4c48daa18988d72ab686fa" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38589 } }
I20260808 23:03:41.973585 20620 catalog_manager.cc:5495] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to be6196364a8a4cfda75b960699f8e7a8 (127.23.194.2). New cstate: current_term: 1 leader_uuid: "be6196364a8a4cfda75b960699f8e7a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be6196364a8a4cfda75b960699f8e7a8" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 35687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c78388c05a134d6cad559178c7e0dd76" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b2f91380e4c48daa18988d72ab686fa" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38589 } health_report { overall_health: UNKNOWN } } }
W20260808 23:03:41.981850 21334 tablet.cc:2369] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:03:41.990563 20914 tablet.cc:2369] T d0aa9bd08bbc48dd9f017c50582a75cd P c78388c05a134d6cad559178c7e0dd76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:03:42.033700 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:42.031413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39592 (local address 127.23.194.2:35687)
0808 23:03:42.031469 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:42.031471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:42.031479 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:03:42.031522 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:42.031523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:42.031546 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:42.031608 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:03:42.031751 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:42.032209 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:42.032935 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:42.033120 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:42.033173 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:42.033341 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:03:42.033452 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:03:42.033536 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:42.033590 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 23:03:42.033599 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 23:03:42.035441 21255 raft_consensus.cc:1261] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa [term 1 FOLLOWER]: Refusing update from remote peer be6196364a8a4cfda75b960699f8e7a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:03:42.035444 20832 raft_consensus.cc:1261] T d0aa9bd08bbc48dd9f017c50582a75cd P c78388c05a134d6cad559178c7e0dd76 [term 1 FOLLOWER]: Refusing update from remote peer be6196364a8a4cfda75b960699f8e7a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:03:42.035796 21371 consensus_queue.cc:1035] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b2f91380e4c48daa18988d72ab686fa" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 38589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:03:42.035921 21368 consensus_queue.cc:1035] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78388c05a134d6cad559178c7e0dd76" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 41123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:03:42.038323 21375 mvcc.cc:205] Tried to move back new op lower bound from 7316398989454614528 to 7316398989200506880. Current Snapshot: MvccSnapshot[applied={T|T < 7316398989454614528}]
I20260808 23:03:42.039263 21380 mvcc.cc:205] Tried to move back new op lower bound from 7316398989454614528 to 7316398989200506880. Current Snapshot: MvccSnapshot[applied={T|T < 7316398989454614528 or (T in {7316398989454614528})}]
I20260808 23:03:42.040594 20620 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:03:42.040674 20620 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:03:42.041474 20620 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db7d1683607a47179c7257c4249824bd: Sending DeleteTablet for 3 replicas of tablet d0aa9bd08bbc48dd9f017c50582a75cd
I20260808 23:03:42.041774 21023 tablet_service.cc:1514] Processing DeleteTablet for tablet d0aa9bd08bbc48dd9f017c50582a75cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:03:42 UTC) from {username='slave'} at 127.0.0.1:39578
I20260808 23:03:42.041782 21229 tablet_service.cc:1514] Processing DeleteTablet for tablet d0aa9bd08bbc48dd9f017c50582a75cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:03:42 UTC) from {username='slave'} at 127.0.0.1:58528
I20260808 23:03:42.041824 20807 tablet_service.cc:1514] Processing DeleteTablet for tablet d0aa9bd08bbc48dd9f017c50582a75cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:03:42 UTC) from {username='slave'} at 127.0.0.1:44806
I20260808 23:03:42.042045 21390 tablet_replica.cc:330] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8: stopping tablet replica
I20260808 23:03:42.042145 21390 raft_consensus.cc:2229] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:03:42.042040 21392 tablet_replica.cc:330] T d0aa9bd08bbc48dd9f017c50582a75cd P c78388c05a134d6cad559178c7e0dd76: stopping tablet replica
I20260808 23:03:42.042246 21390 raft_consensus.cc:2258] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:03:42.042264 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 20706
I20260808 23:03:42.052059 21390 ts_tablet_manager.cc:1905] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:03:42.054122 21390 ts_tablet_manager.cc:1918] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:03:42.054209 21390 log.cc:1196] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/d0aa9bd08bbc48dd9f017c50582a75cd
I20260808 23:03:42.054515 21390 ts_tablet_manager.cc:1939] T d0aa9bd08bbc48dd9f017c50582a75cd P be6196364a8a4cfda75b960699f8e7a8: Deleting consensus metadata
I20260808 23:03:42.059370 20601 catalog_manager.cc:4841] TS be6196364a8a4cfda75b960699f8e7a8 (127.23.194.2:35687): tablet d0aa9bd08bbc48dd9f017c50582a75cd (table test-table [id=a3bcd0d1bc9146af92b9173bbfb9d731]) successfully deleted
I20260808 23:03:42.059581 21391 tablet_replica.cc:330] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa: stopping tablet replica
I20260808 23:03:42.059710 21391 raft_consensus.cc:2229] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:03:42.059831 21391 raft_consensus.cc:2258] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:03:42.060350 21391 ts_tablet_manager.cc:1905] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:03:42.061992 21391 ts_tablet_manager.cc:1918] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:03:42.062070 21391 log.cc:1196] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/d0aa9bd08bbc48dd9f017c50582a75cd
I20260808 23:03:42.062325 21391 ts_tablet_manager.cc:1939] T d0aa9bd08bbc48dd9f017c50582a75cd P 7b2f91380e4c48daa18988d72ab686fa: Deleting consensus metadata
I20260808 23:03:42.062754 20606 catalog_manager.cc:4841] TS 7b2f91380e4c48daa18988d72ab686fa (127.23.194.3:38589): tablet d0aa9bd08bbc48dd9f017c50582a75cd (table test-table [id=a3bcd0d1bc9146af92b9173bbfb9d731]) successfully deleted
W20260808 23:03:42.063571 20601 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.194.1:41123 (error 108)
I20260808 23:03:42.063827 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 20918
I20260808 23:03:42.065469 21343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:42.063906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:41123 (local address 127.0.0.1:44814)
0808 23:03:42.065343 (+ 1437us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:42.065431 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.1:41123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1407}
W20260808 23:03:42.071640 20601 catalog_manager.cc:4584] TS c78388c05a134d6cad559178c7e0dd76 (127.23.194.1:41123): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d0aa9bd08bbc48dd9f017c50582a75cd: Network error: Client connection negotiation failed: client connection to 127.23.194.1:41123: connect: Connection refused (error 111)
I20260808 23:03:42.080138 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 21136
I20260808 23:03:42.090042 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 20577
2026-08-08T23:03:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (895 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:03:42 dist-test-slave-f0tn krb5kdc[21397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:03:42 dist-test-slave-f0tn krb5kdc[21397](info): set up 2 sockets
Aug 08 23:03:42 dist-test-slave-f0tn krb5kdc[21397](info): commencing operation
krb5kdc: starting...
W20260808 23:03:43.237353 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:03:43 dist-test-slave-f0tn krb5kdc[21397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:03:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:03:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:03:43.349712 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:34899
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:32961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:34899
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:43.432515 21417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:43.432731 21417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:43.432755 21417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:43.434019 21417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:03:43.434070 21417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:43.434087 21417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:43.434100 21417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:03:43.434116 21417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:03:43.435662 21417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:32961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:34899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:34899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:43.435917 21417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:43.436151 21417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:43.441004 21423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:43.440958 21426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:43.441094 21417 server_base.cc:1034] running on GCE node
W20260808 23:03:43.440956 21424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:43.441478 21417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:43.441781 21417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:43.442916 21417 hybrid_clock.cc:648] HybridClock initialized: now 1786230223442898 us; error 39 us; skew 500 ppm
Aug 08 23:03:43 dist-test-slave-f0tn krb5kdc[21397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230223, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:43.445284 21417 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:03:43.445704 21417 webserver.cc:460] Webserver started at http://127.23.194.62:34491/ using document root <none> and password file <none>
I20260808 23:03:43.445906 21417 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:43.445953 21417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:43.446077 21417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:43.447072 21417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "ccc7af19488c4cbe90d0585368e3f468"
format_stamp: "Formatted at 2026-08-08 23:03:43 on dist-test-slave-f0tn"
server_key: "f106d9b0f70fad99429f44e17ae44e18"
server_key_iv: "61a505b7e3eebe860ad097aa39995141"
server_key_version: "encryptionkey@0"
I20260808 23:03:43.447393 21417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "ccc7af19488c4cbe90d0585368e3f468"
format_stamp: "Formatted at 2026-08-08 23:03:43 on dist-test-slave-f0tn"
server_key: "f106d9b0f70fad99429f44e17ae44e18"
server_key_iv: "61a505b7e3eebe860ad097aa39995141"
server_key_version: "encryptionkey@0"
I20260808 23:03:43.448630 21417 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 23:03:43.449265 21436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:43.449446 21417 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:03:43.449532 21417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "ccc7af19488c4cbe90d0585368e3f468"
format_stamp: "Formatted at 2026-08-08 23:03:43 on dist-test-slave-f0tn"
server_key: "f106d9b0f70fad99429f44e17ae44e18"
server_key_iv: "61a505b7e3eebe860ad097aa39995141"
server_key_version: "encryptionkey@0"
I20260808 23:03:43.449599 21417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:43.461202 21417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:43.463166 21417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:43.463295 21417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:43.467078 21417 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:34899
I20260808 23:03:43.467114 21511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:34899 every 8 connection(s)
I20260808 23:03:43.467411 21417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:03:43.468236 21512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:43.470410 21512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468: Bootstrap starting.
I20260808 23:03:43.470917 21512 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:43.471182 21512 log.cc:824] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:43.471997 21512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468: No bootstrap required, opened a new log
I20260808 23:03:43.473764 21512 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccc7af19488c4cbe90d0585368e3f468" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34899 } }
I20260808 23:03:43.473894 21512 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:43.473915 21512 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccc7af19488c4cbe90d0585368e3f468, State: Initialized, Role: FOLLOWER
I20260808 23:03:43.474030 21512 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccc7af19488c4cbe90d0585368e3f468" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34899 } }
I20260808 23:03:43.474099 21512 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:03:43.474124 21512 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:03:43.474177 21512 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:43.474404 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 21417
I20260808 23:03:43.474529 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:03:43.474704 24328 external_mini_cluster.cc:1428] Setting key db2cf39add2587b368b56ecb50ce6432
I20260808 23:03:43.474758 21512 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccc7af19488c4cbe90d0585368e3f468" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34899 } }
I20260808 23:03:43.474874 21512 leader_election.cc:302] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccc7af19488c4cbe90d0585368e3f468; no voters:
I20260808 23:03:43.475029 21512 leader_election.cc:288] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:03:43.475067 21516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:43.475281 21512 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:03:43.475297 21516 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468 [term 1 LEADER]: Becoming Leader. State: Replica: ccc7af19488c4cbe90d0585368e3f468, State: Running, Role: LEADER
I20260808 23:03:43.475390 21516 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccc7af19488c4cbe90d0585368e3f468" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34899 } }
I20260808 23:03:43.477205 21517 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccc7af19488c4cbe90d0585368e3f468. Latest consensus state: current_term: 1 leader_uuid: "ccc7af19488c4cbe90d0585368e3f468" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccc7af19488c4cbe90d0585368e3f468" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34899 } } }
I20260808 23:03:43.477336 21521 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccc7af19488c4cbe90d0585368e3f468" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccc7af19488c4cbe90d0585368e3f468" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 34899 } } }
I20260808 23:03:43.478096 21517 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:43.478102 21521 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468 [sys.catalog]: This master's current role is: LEADER
W20260808 23:03:43.477456 21538 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:03:43.478355 21538 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:03:43.478426 21542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:03:43.478583 21542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:03:43.480132 21542 catalog_manager.cc:1347] Generated new cluster ID: 58bec9a7ba1449a09cd04e3915ffa022
I20260808 23:03:43.480188 21542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 23:03:43 dist-test-slave-f0tn krb5kdc[21397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:03:43.485173 21519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:43.475094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39320 (local address 127.23.194.62:34899)
0808 23:03:43.475282 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:43.475291 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:43.475565 (+ 274us) server_negotiation.cc:407] Connection header received
0808 23:03:43.475684 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:43.475690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:43.475745 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:43.475886 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:43.476647 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.477873 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.478937 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.479537 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.479656 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:43.481850 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:43.481854 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:43.481854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:43.481866 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:43.483841 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:43.484165 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:43.484168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:43.484169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:43.484260 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:43.484431 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:43.484432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:43.484432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:43.484653 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:43.484792 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:43.484889 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 23:03:43.485018 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":101,"threads_started":1}
I20260808 23:03:43.495398 21542 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:03:43.496107 21542 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:03:43.500500 21542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468: Generated new TSK 0
I20260808 23:03:43.500674 21542 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
Entry for principal kudu/127.23.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab.
I20260808 23:03:43.531754 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34899
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:32961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:43.611208 21549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:43.611402 21549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:43.611423 21549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:43.612479 21549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:43.612510 21549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:43.612535 21549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:03:43.613932 21549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:32961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.1.keytab
--principal=kudu/127.23.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:43.614104 21549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:43.614279 21549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:43.619027 21558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:43.619050 21562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:43.619107 21560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:43.619247 21549 server_base.cc:1034] running on GCE node
I20260808 23:03:43.619428 21549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:43.619621 21549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:43.620746 21549 hybrid_clock.cc:648] HybridClock initialized: now 1786230223620734 us; error 30 us; skew 500 ppm
Aug 08 23:03:43 dist-test-slave-f0tn krb5kdc[21397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230223, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:43.623102 21549 init.cc:377] Logged in from keytab as kudu/127.23.194.1@KRBTEST.COM (short username kudu)
I20260808 23:03:43.623522 21549 webserver.cc:460] Webserver started at http://127.23.194.1:39269/ using document root <none> and password file <none>
I20260808 23:03:43.623710 21549 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:43.623752 21549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:43.623872 21549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:43.624810 21549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "cba5be0417d84e41be4fff71f15cb255"
format_stamp: "Formatted at 2026-08-08 23:03:43 on dist-test-slave-f0tn"
server_key: "f525a16ef6e592aeb7b0140a40795717"
server_key_iv: "3996869c8c056040c070aa5b8a22b94b"
server_key_version: "encryptionkey@0"
I20260808 23:03:43.625120 21549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "cba5be0417d84e41be4fff71f15cb255"
format_stamp: "Formatted at 2026-08-08 23:03:43 on dist-test-slave-f0tn"
server_key: "f525a16ef6e592aeb7b0140a40795717"
server_key_iv: "3996869c8c056040c070aa5b8a22b94b"
server_key_version: "encryptionkey@0"
I20260808 23:03:43.626390 21549 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:03:43.627012 21570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:43.627135 21549 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:43.627221 21549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "cba5be0417d84e41be4fff71f15cb255"
format_stamp: "Formatted at 2026-08-08 23:03:43 on dist-test-slave-f0tn"
server_key: "f525a16ef6e592aeb7b0140a40795717"
server_key_iv: "3996869c8c056040c070aa5b8a22b94b"
server_key_version: "encryptionkey@0"
I20260808 23:03:43.627285 21549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:43.643102 21549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:43.644802 21549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:43.644924 21549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:43.645124 21549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:43.645493 21549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:43.645531 21549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:43.645555 21549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:43.645576 21549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:43.650977 21549 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:36715
I20260808 23:03:43.651037 21747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:36715 every 8 connection(s)
I20260808 23:03:43.651324 21549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:03:43.656446 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 21549
I20260808 23:03:43.656570 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:03:43.656788 24328 external_mini_cluster.cc:1428] Setting key df0f8b44dccfb8849d9a3e206a537d3d
Aug 08 23:03:43 dist-test-slave-f0tn krb5kdc[21397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230223, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.1@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:03:43.659977 21519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:43.651970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:54855 (local address 127.23.194.62:34899)
0808 23:03:43.652038 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:43.652040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:43.652213 (+ 173us) server_negotiation.cc:407] Connection header received
0808 23:03:43.652656 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:43.652657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:43.652670 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:43.652721 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:43.653667 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.654165 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.655052 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.655243 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.655285 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:43.657411 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:43.657413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:43.657413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:43.657422 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:43.658638 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:43.659071 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:43.659072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:43.659072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:43.659133 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:43.659393 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:43.659394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:43.659394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:43.659522 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:43.659586 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:43.659798 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 23:03:43.659886 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 23:03:43.660035 21750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:43.651939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34899 (local address 127.23.194.1:54855)
0808 23:03:43.652091 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:43.652131 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:43.652579 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:43.652786 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:43.652790 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:43.653400 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:43.653594 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:43.653599 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.654230 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.654230 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:43.654994 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:43.654997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.655294 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.655295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:43.655391 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:43.655791 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:03:43.655799 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:43.657297 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:43.658715 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:43.658723 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:43.658725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:43.659002 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:43.659202 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:43.659203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:43.659203 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:43.659336 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:43.659576 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:43.659579 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:43.659686 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 23:03:43.659761 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 23:03:43.659878 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260808 23:03:43.660532 21748 heartbeater.cc:347] Connected to a master server at 127.23.194.62:34899
I20260808 23:03:43.660624 21748 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:43.660796 21748 heartbeater.cc:511] Master 127.23.194.62:34899 requested a full tablet report, sending...
I20260808 23:03:43.661336 21461 ts_manager.cc:194] Registered new tserver with Master: cba5be0417d84e41be4fff71f15cb255 (127.23.194.1:36715)
I20260808 23:03:43.662241 21461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.1@KRBTEST.COM'} at 127.23.194.1:54855
WARNING: no policy specified for kudu/127.23.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
Entry for principal kudu/127.23.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab.
I20260808 23:03:43.680852 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34899
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:32961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:43.759845 21757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:43.760035 21757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:43.760056 21757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:43.761175 21757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:43.761215 21757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:43.761245 21757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:03:43.762719 21757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:32961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.2.keytab
--principal=kudu/127.23.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:43.762929 21757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:43.763139 21757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:43.768527 21766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:43.768610 21757 server_base.cc:1034] running on GCE node
W20260808 23:03:43.768543 21765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:43.768535 21769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:43.768909 21757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:43.769126 21757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:43.770267 21757 hybrid_clock.cc:648] HybridClock initialized: now 1786230223770256 us; error 29 us; skew 500 ppm
Aug 08 23:03:43 dist-test-slave-f0tn krb5kdc[21397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230223, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:43.772521 21757 init.cc:377] Logged in from keytab as kudu/127.23.194.2@KRBTEST.COM (short username kudu)
I20260808 23:03:43.772944 21757 webserver.cc:460] Webserver started at http://127.23.194.2:41861/ using document root <none> and password file <none>
I20260808 23:03:43.773170 21757 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:43.773218 21757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:43.773340 21757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:43.774364 21757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "9079ab3a642f4673958e12806a0f49a5"
format_stamp: "Formatted at 2026-08-08 23:03:43 on dist-test-slave-f0tn"
server_key: "3f6416b3944f38ad921b08faf07d0d2e"
server_key_iv: "460c5789e2dbbdc22dd40c808ce59fb6"
server_key_version: "encryptionkey@0"
I20260808 23:03:43.774684 21757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "9079ab3a642f4673958e12806a0f49a5"
format_stamp: "Formatted at 2026-08-08 23:03:43 on dist-test-slave-f0tn"
server_key: "3f6416b3944f38ad921b08faf07d0d2e"
server_key_iv: "460c5789e2dbbdc22dd40c808ce59fb6"
server_key_version: "encryptionkey@0"
I20260808 23:03:43.775911 21757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:03:43.776587 21776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:43.776758 21757 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:43.776849 21757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "9079ab3a642f4673958e12806a0f49a5"
format_stamp: "Formatted at 2026-08-08 23:03:43 on dist-test-slave-f0tn"
server_key: "3f6416b3944f38ad921b08faf07d0d2e"
server_key_iv: "460c5789e2dbbdc22dd40c808ce59fb6"
server_key_version: "encryptionkey@0"
I20260808 23:03:43.776914 21757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:43.802352 21757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:43.804023 21757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:43.804167 21757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:43.804388 21757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:43.804690 21757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:43.804725 21757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:43.804754 21757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:43.804771 21757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:43.810563 21757 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:44115
I20260808 23:03:43.810604 21931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:44115 every 8 connection(s)
I20260808 23:03:43.810911 21757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:03:43.815460 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 21757
I20260808 23:03:43.815562 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:03:43.815757 24328 external_mini_cluster.cc:1428] Setting key 154e3c99be651287b83122d0da572704
Aug 08 23:03:43 dist-test-slave-f0tn krb5kdc[21397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230223, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.2@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:03:43.819454 21519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:43.811538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:45825 (local address 127.23.194.62:34899)
0808 23:03:43.811621 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:43.811624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:43.811808 (+ 184us) server_negotiation.cc:407] Connection header received
0808 23:03:43.812272 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:43.812273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:43.812283 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:43.812334 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:43.813187 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.813738 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.814662 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.814849 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.814894 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:43.817058 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:43.817060 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:43.817060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:43.817068 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:43.818241 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:43.818592 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:43.818593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:43.818593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:43.818646 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:43.818867 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:43.818868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:43.818869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:43.818972 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:43.819029 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:43.819283 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 23:03:43.819366 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 23:03:43.819602 21937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:43.811511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34899 (local address 127.23.194.2:45825)
0808 23:03:43.811679 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:43.811714 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:43.812195 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:43.812387 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:43.812390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:43.812938 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:43.813120 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:43.813124 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.813810 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.813811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:43.814586 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:43.814588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.814902 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.814903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:43.814993 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:43.815436 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:03:43.815443 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:43.816970 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:43.818312 (+ 1342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:43.818319 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:43.818321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:43.818525 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:43.818702 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:43.818703 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:43.818703 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:43.818810 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:43.819028 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:43.819035 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:43.819166 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 23:03:43.819252 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 23:03:43.819399 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260808 23:03:43.820009 21932 heartbeater.cc:347] Connected to a master server at 127.23.194.62:34899
I20260808 23:03:43.820106 21932 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:43.820290 21932 heartbeater.cc:511] Master 127.23.194.62:34899 requested a full tablet report, sending...
I20260808 23:03:43.820678 21461 ts_manager.cc:194] Registered new tserver with Master: 9079ab3a642f4673958e12806a0f49a5 (127.23.194.2:44115)
I20260808 23:03:43.821028 21461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.2@KRBTEST.COM'} at 127.23.194.2:45825
WARNING: no policy specified for kudu/127.23.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
Entry for principal kudu/127.23.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab.
I20260808 23:03:43.838266 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34899
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:32961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:43.917730 21940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:43.917934 21940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:43.917956 21940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:43.919027 21940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:43.919062 21940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:43.919085 21940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:03:43.920459 21940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:32961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.3.keytab
--principal=kudu/127.23.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:34899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:43.920631 21940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:43.920804 21940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:43.925702 21947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:43.925653 21948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:43.925753 21940 server_base.cc:1034] running on GCE node
W20260808 23:03:43.925789 21954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:43.926065 21940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:43.926297 21940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:43.927435 21940 hybrid_clock.cc:648] HybridClock initialized: now 1786230223927421 us; error 29 us; skew 500 ppm
Aug 08 23:03:43 dist-test-slave-f0tn krb5kdc[21397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230223, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:43.929842 21940 init.cc:377] Logged in from keytab as kudu/127.23.194.3@KRBTEST.COM (short username kudu)
I20260808 23:03:43.930269 21940 webserver.cc:460] Webserver started at http://127.23.194.3:34307/ using document root <none> and password file <none>
I20260808 23:03:43.930461 21940 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:43.930502 21940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:43.930622 21940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:43.931627 21940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "83beec735009427998fe1dd39c3cf3c3"
format_stamp: "Formatted at 2026-08-08 23:03:43 on dist-test-slave-f0tn"
server_key: "0f3874dd3d6054791826488825aa32d6"
server_key_iv: "543b27b5fa4f41ea43588795f5101597"
server_key_version: "encryptionkey@0"
I20260808 23:03:43.931949 21940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "83beec735009427998fe1dd39c3cf3c3"
format_stamp: "Formatted at 2026-08-08 23:03:43 on dist-test-slave-f0tn"
server_key: "0f3874dd3d6054791826488825aa32d6"
server_key_iv: "543b27b5fa4f41ea43588795f5101597"
server_key_version: "encryptionkey@0"
I20260808 23:03:43.933126 21940 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 23:03:43.933820 21963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:43.933984 21940 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:43.934063 21940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "83beec735009427998fe1dd39c3cf3c3"
format_stamp: "Formatted at 2026-08-08 23:03:43 on dist-test-slave-f0tn"
server_key: "0f3874dd3d6054791826488825aa32d6"
server_key_iv: "543b27b5fa4f41ea43588795f5101597"
server_key_version: "encryptionkey@0"
I20260808 23:03:43.934113 21940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:43.948693 21940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:43.950647 21940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:43.950752 21940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:43.950922 21940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:43.951153 21940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:43.951181 21940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:43.951200 21940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:43.951211 21940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:43.956642 21940 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:42105
I20260808 23:03:43.956709 22137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:42105 every 8 connection(s)
I20260808 23:03:43.956957 21940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
Aug 08 23:03:43 dist-test-slave-f0tn krb5kdc[21397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230223, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.3@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:03:43.962514 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 21940
I20260808 23:03:43.962599 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:03:43.962787 24328 external_mini_cluster.cc:1428] Setting key 25125ef7174a7e53320c62a20f8018fc
I20260808 23:03:43.965095 21519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:43.957623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:45627 (local address 127.23.194.62:34899)
0808 23:03:43.957665 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:43.957667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:43.957873 (+ 206us) server_negotiation.cc:407] Connection header received
0808 23:03:43.958316 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:43.958317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:43.958328 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:43.958376 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:43.959200 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.959675 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.960583 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.960756 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.960795 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:43.962652 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:43.962653 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:43.962654 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:43.962662 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:43.964044 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:43.964375 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:43.964376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:43.964376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:43.964427 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:43.964609 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:43.964610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:43.964610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:43.964738 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:43.964790 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:43.964978 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 23:03:43.965034 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 23:03:43.965272 22141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:43.957600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:34899 (local address 127.23.194.3:45627)
0808 23:03:43.957765 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:43.957797 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:43.958242 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:43.958424 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:43.958427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:43.958949 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:43.959123 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:43.959127 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.959747 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.959748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:43.960523 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:43.960525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.960807 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.960807 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:43.960899 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:43.961250 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:03:43.961258 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:43.962543 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:43.964111 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:43.964117 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:43.964119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:43.964321 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:43.964473 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:03:43.964474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:03:43.964474 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:03:43.964559 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:03:43.964786 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:43.964789 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:43.964891 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 23:03:43.964964 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 23:03:43.965075 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
I20260808 23:03:43.965680 22138 heartbeater.cc:347] Connected to a master server at 127.23.194.62:34899
I20260808 23:03:43.965761 22138 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:43.965898 22138 heartbeater.cc:511] Master 127.23.194.62:34899 requested a full tablet report, sending...
I20260808 23:03:43.966217 21461 ts_manager.cc:194] Registered new tserver with Master: 83beec735009427998fe1dd39c3cf3c3 (127.23.194.3:42105)
I20260808 23:03:43.966543 21461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.194.3@KRBTEST.COM'} at 127.23.194.3:45627
I20260808 23:03:43.974290 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:03:43.980957 21519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:43.975514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39334 (local address 127.23.194.62:34899)
0808 23:03:43.975549 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:43.975551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:43.975717 (+ 166us) server_negotiation.cc:407] Connection header received
0808 23:03:43.975774 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:43.975775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:43.975785 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:43.975829 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:43.976373 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.976874 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.977703 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.977877 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.977937 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:43.978957 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:43.978958 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:43.978959 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:43.978966 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:43.979946 (+ 980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:43.980240 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:43.980241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:43.980241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:43.980288 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:43.980444 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:43.980444 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:43.980445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:43.980609 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:43.980695 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:43.980802 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 23:03:43.980872 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 23:03:43.982227 21461 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:03:43.983629 21461 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:03:43.990252 22154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:43.985583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46106 (local address 127.23.194.3:42105)
0808 23:03:43.985752 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:43.985758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:43.985947 (+ 189us) server_negotiation.cc:407] Connection header received
0808 23:03:43.985990 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:43.985992 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:43.986007 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:43.986078 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:43.986391 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.987504 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.989120 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.989853 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.989929 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:43.989988 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:43.990137 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 23:03:43.990177 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
I20260808 23:03:43.990242 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:43.985511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:42105 (local address 127.0.0.1:46106)
0808 23:03:43.985846 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:43.985855 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:43.985896 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:43.986121 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:43.986124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:43.986127 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:43.986318 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:43.986321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.987617 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.987618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:43.989023 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:43.989026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.989945 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.989946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:43.990036 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:43.990049 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 23:03:43.990117 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 23:03:43.990171 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":48,"thread_start_us":48,"threads_started":1}
I20260808 23:03:43.990411 22157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:43.985753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41314 (local address 127.23.194.2:44115)
0808 23:03:43.985934 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:43.985938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:43.985949 (+ 11us) server_negotiation.cc:407] Connection header received
0808 23:03:43.985986 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:43.985988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:43.986005 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:43.986087 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:43.986502 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.987564 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.989453 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.989938 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.990006 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:43.990063 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:43.990295 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 23:03:43.990339 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":105,"threads_started":1}
I20260808 23:03:43.990469 22152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:43.985503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:44115 (local address 127.0.0.1:41314)
0808 23:03:43.985653 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:43.985689 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:43.985751 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:43.986409 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:43.986410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:43.986412 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:43.986443 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:43.986444 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.987675 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.987676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:43.989023 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:43.989026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.990122 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.990122 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:43.990177 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:43.990181 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:03:43.990225 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 23:03:43.990254 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":39,"thread_start_us":52,"threads_started":1}
I20260808 23:03:43.991151 22042 tablet_service.cc:1467] Processing CreateTablet for tablet 0a3a5b242e6f49008154b206f95fbe0b (DEFAULT_TABLE table=test-table [id=ff19cba1eebc4b6a9ed33ec0cd3957af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:43.991146 21849 tablet_service.cc:1467] Processing CreateTablet for tablet 0a3a5b242e6f49008154b206f95fbe0b (DEFAULT_TABLE table=test-table [id=ff19cba1eebc4b6a9ed33ec0cd3957af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:43.991495 21849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a3a5b242e6f49008154b206f95fbe0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:43.991495 22042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a3a5b242e6f49008154b206f95fbe0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:43.994802 22160 tablet_bootstrap.cc:492] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5: Bootstrap starting.
I20260808 23:03:43.995010 22159 tablet_bootstrap.cc:492] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3: Bootstrap starting.
I20260808 23:03:43.995533 22155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:43.985680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:36715 (local address 127.0.0.1:33748)
0808 23:03:43.985937 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:43.985946 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:43.985986 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:43.987788 (+ 1802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:43.987790 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:43.987792 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:43.987884 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:43.987888 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.988847 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.988848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:43.989898 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:43.989901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.995303 (+ 5402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.995304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:43.995365 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:43.995369 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:03:43.995423 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 23:03:43.995463 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":30,"thread_start_us":47,"threads_started":1}
I20260808 23:03:43.995538 22160 tablet_bootstrap.cc:654] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:43.995548 22158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:43.985852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33748 (local address 127.23.194.1:36715)
0808 23:03:43.986339 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:43.986344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:43.986355 (+ 11us) server_negotiation.cc:407] Connection header received
0808 23:03:43.987687 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:43.987690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:43.987706 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:43.987776 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:43.987949 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.988761 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.989981 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:43.995232 (+ 5251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:43.995301 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:43.995358 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:43.995436 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 23:03:43.995473 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":89,"threads_started":1}
I20260808 23:03:43.995678 22159 tablet_bootstrap.cc:654] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:43.996158 22160 log.cc:824] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:43.996210 22159 log.cc:824] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:43.996434 21646 tablet_service.cc:1467] Processing CreateTablet for tablet 0a3a5b242e6f49008154b206f95fbe0b (DEFAULT_TABLE table=test-table [id=ff19cba1eebc4b6a9ed33ec0cd3957af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:43.996793 21646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a3a5b242e6f49008154b206f95fbe0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:43.996985 22159 tablet_bootstrap.cc:492] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3: No bootstrap required, opened a new log
I20260808 23:03:43.997061 22159 ts_tablet_manager.cc:1397] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 23:03:43.997968 22160 tablet_bootstrap.cc:492] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5: No bootstrap required, opened a new log
I20260808 23:03:43.998044 22160 ts_tablet_manager.cc:1397] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 23:03:43.998883 22159 raft_consensus.cc:348] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9079ab3a642f4673958e12806a0f49a5" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44115 } } peers { permanent_uuid: "83beec735009427998fe1dd39c3cf3c3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 42105 } } peers { permanent_uuid: "cba5be0417d84e41be4fff71f15cb255" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36715 } }
I20260808 23:03:43.999061 22159 raft_consensus.cc:374] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:43.999126 22159 raft_consensus.cc:729] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83beec735009427998fe1dd39c3cf3c3, State: Initialized, Role: FOLLOWER
I20260808 23:03:43.999276 22159 consensus_queue.cc:260] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9079ab3a642f4673958e12806a0f49a5" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44115 } } peers { permanent_uuid: "83beec735009427998fe1dd39c3cf3c3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 42105 } } peers { permanent_uuid: "cba5be0417d84e41be4fff71f15cb255" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36715 } }
I20260808 23:03:43.999533 22163 tablet_bootstrap.cc:492] T 0a3a5b242e6f49008154b206f95fbe0b P cba5be0417d84e41be4fff71f15cb255: Bootstrap starting.
I20260808 23:03:43.999554 22159 ts_tablet_manager.cc:1428] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 23:03:43.999671 22138 heartbeater.cc:503] Master 127.23.194.62:34899 was elected leader, sending a full tablet report...
I20260808 23:03:43.999919 22160 raft_consensus.cc:348] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9079ab3a642f4673958e12806a0f49a5" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44115 } } peers { permanent_uuid: "83beec735009427998fe1dd39c3cf3c3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 42105 } } peers { permanent_uuid: "cba5be0417d84e41be4fff71f15cb255" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36715 } }
I20260808 23:03:44.000077 22160 raft_consensus.cc:374] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:44.000111 22160 raft_consensus.cc:729] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9079ab3a642f4673958e12806a0f49a5, State: Initialized, Role: FOLLOWER
I20260808 23:03:44.000226 22160 consensus_queue.cc:260] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9079ab3a642f4673958e12806a0f49a5" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44115 } } peers { permanent_uuid: "83beec735009427998fe1dd39c3cf3c3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 42105 } } peers { permanent_uuid: "cba5be0417d84e41be4fff71f15cb255" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36715 } }
I20260808 23:03:44.000490 22160 ts_tablet_manager.cc:1428] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 23:03:44.000517 21932 heartbeater.cc:503] Master 127.23.194.62:34899 was elected leader, sending a full tablet report...
I20260808 23:03:44.000664 22163 tablet_bootstrap.cc:654] T 0a3a5b242e6f49008154b206f95fbe0b P cba5be0417d84e41be4fff71f15cb255: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:44.001266 22163 log.cc:824] T 0a3a5b242e6f49008154b206f95fbe0b P cba5be0417d84e41be4fff71f15cb255: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:44.001937 22163 tablet_bootstrap.cc:492] T 0a3a5b242e6f49008154b206f95fbe0b P cba5be0417d84e41be4fff71f15cb255: No bootstrap required, opened a new log
I20260808 23:03:44.001996 22163 ts_tablet_manager.cc:1397] T 0a3a5b242e6f49008154b206f95fbe0b P cba5be0417d84e41be4fff71f15cb255: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 23:03:44.003286 22163 raft_consensus.cc:348] T 0a3a5b242e6f49008154b206f95fbe0b P cba5be0417d84e41be4fff71f15cb255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9079ab3a642f4673958e12806a0f49a5" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44115 } } peers { permanent_uuid: "83beec735009427998fe1dd39c3cf3c3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 42105 } } peers { permanent_uuid: "cba5be0417d84e41be4fff71f15cb255" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36715 } }
I20260808 23:03:44.003409 22163 raft_consensus.cc:374] T 0a3a5b242e6f49008154b206f95fbe0b P cba5be0417d84e41be4fff71f15cb255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:44.003443 22163 raft_consensus.cc:729] T 0a3a5b242e6f49008154b206f95fbe0b P cba5be0417d84e41be4fff71f15cb255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cba5be0417d84e41be4fff71f15cb255, State: Initialized, Role: FOLLOWER
I20260808 23:03:44.003551 22163 consensus_queue.cc:260] T 0a3a5b242e6f49008154b206f95fbe0b P cba5be0417d84e41be4fff71f15cb255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9079ab3a642f4673958e12806a0f49a5" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44115 } } peers { permanent_uuid: "83beec735009427998fe1dd39c3cf3c3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 42105 } } peers { permanent_uuid: "cba5be0417d84e41be4fff71f15cb255" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36715 } }
I20260808 23:03:44.003777 22163 ts_tablet_manager.cc:1428] T 0a3a5b242e6f49008154b206f95fbe0b P cba5be0417d84e41be4fff71f15cb255: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 23:03:44.003814 21748 heartbeater.cc:503] Master 127.23.194.62:34899 was elected leader, sending a full tablet report...
W20260808 23:03:44.061471 21935 tablet.cc:2369] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:03:44.152016 21749 tablet.cc:2369] T 0a3a5b242e6f49008154b206f95fbe0b P cba5be0417d84e41be4fff71f15cb255: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:03:44.152731 22165 raft_consensus.cc:482] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:03:44.152843 22165 raft_consensus.cc:504] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9079ab3a642f4673958e12806a0f49a5" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44115 } } peers { permanent_uuid: "83beec735009427998fe1dd39c3cf3c3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 42105 } } peers { permanent_uuid: "cba5be0417d84e41be4fff71f15cb255" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36715 } }
I20260808 23:03:44.153189 22165 leader_election.cc:288] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83beec735009427998fe1dd39c3cf3c3 (127.23.194.3:42105), cba5be0417d84e41be4fff71f15cb255 (127.23.194.1:36715)
I20260808 23:03:44.156028 22158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:44.153557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:43917 (local address 127.23.194.1:36715)
0808 23:03:44.153622 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:44.153624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:44.153742 (+ 118us) server_negotiation.cc:407] Connection header received
0808 23:03:44.153773 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:44.153774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:44.153784 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:44.153832 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:44.153976 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.154445 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.155357 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.155710 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.155768 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:44.155777 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:44.155919 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 23:03:44.155958 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 23:03:44.156044 22154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:44.153462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:60247 (local address 127.23.194.3:42105)
0808 23:03:44.153510 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:44.153512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:44.153521 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:03:44.153548 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:44.153549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:44.153557 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:44.153606 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:44.153831 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.154365 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.155347 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.155721 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.155787 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:44.155796 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:44.155940 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 23:03:44.155976 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 23:03:44.156049 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:44.153479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:36715 (local address 127.23.194.2:43917)
0808 23:03:44.153670 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:44.153679 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:44.153739 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:44.153849 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:44.153850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:44.153851 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:44.153922 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:44.153924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.154516 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.154517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:44.155270 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:44.155272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.155782 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.155784 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:44.155852 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:44.155856 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:03:44.155899 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 23:03:44.155934 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":125,"threads_started":1}
I20260808 23:03:44.156052 21937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:44.153349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:42105 (local address 127.23.194.2:60247)
0808 23:03:44.153409 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:44.153423 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:44.153485 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:44.153675 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:44.153676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:44.153678 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:44.153743 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:44.153745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.154441 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.154442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:44.155253 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:44.155256 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.155798 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.155799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:44.155874 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:44.155876 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 23:03:44.155926 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 23:03:44.155962 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":43}
I20260808 23:03:44.156381 21680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a3a5b242e6f49008154b206f95fbe0b" candidate_uuid: "9079ab3a642f4673958e12806a0f49a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cba5be0417d84e41be4fff71f15cb255" is_pre_election: true
I20260808 23:03:44.156517 21680 raft_consensus.cc:2454] T 0a3a5b242e6f49008154b206f95fbe0b P cba5be0417d84e41be4fff71f15cb255 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9079ab3a642f4673958e12806a0f49a5 in term 0.
I20260808 23:03:44.156711 21784 leader_election.cc:302] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9079ab3a642f4673958e12806a0f49a5, cba5be0417d84e41be4fff71f15cb255; no voters:
I20260808 23:03:44.156762 22072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a3a5b242e6f49008154b206f95fbe0b" candidate_uuid: "9079ab3a642f4673958e12806a0f49a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83beec735009427998fe1dd39c3cf3c3" is_pre_election: true
I20260808 23:03:44.156831 22165 raft_consensus.cc:2781] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:03:44.156880 22165 raft_consensus.cc:482] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:03:44.156910 22165 raft_consensus.cc:3037] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:44.156910 22072 raft_consensus.cc:2454] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9079ab3a642f4673958e12806a0f49a5 in term 0.
I20260808 23:03:44.157673 22165 raft_consensus.cc:504] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9079ab3a642f4673958e12806a0f49a5" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44115 } } peers { permanent_uuid: "83beec735009427998fe1dd39c3cf3c3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 42105 } } peers { permanent_uuid: "cba5be0417d84e41be4fff71f15cb255" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36715 } }
I20260808 23:03:44.157788 22165 leader_election.cc:288] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [CANDIDATE]: Term 1 election: Requested vote from peers 83beec735009427998fe1dd39c3cf3c3 (127.23.194.3:42105), cba5be0417d84e41be4fff71f15cb255 (127.23.194.1:36715)
I20260808 23:03:44.157972 22072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a3a5b242e6f49008154b206f95fbe0b" candidate_uuid: "9079ab3a642f4673958e12806a0f49a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83beec735009427998fe1dd39c3cf3c3"
I20260808 23:03:44.157987 21680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a3a5b242e6f49008154b206f95fbe0b" candidate_uuid: "9079ab3a642f4673958e12806a0f49a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cba5be0417d84e41be4fff71f15cb255"
I20260808 23:03:44.158054 21680 raft_consensus.cc:3037] T 0a3a5b242e6f49008154b206f95fbe0b P cba5be0417d84e41be4fff71f15cb255 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:44.158054 22072 raft_consensus.cc:3037] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:44.158903 21680 raft_consensus.cc:2454] T 0a3a5b242e6f49008154b206f95fbe0b P cba5be0417d84e41be4fff71f15cb255 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9079ab3a642f4673958e12806a0f49a5 in term 1.
I20260808 23:03:44.158903 22072 raft_consensus.cc:2454] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9079ab3a642f4673958e12806a0f49a5 in term 1.
I20260808 23:03:44.159053 21784 leader_election.cc:302] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83beec735009427998fe1dd39c3cf3c3, 9079ab3a642f4673958e12806a0f49a5; no voters:
I20260808 23:03:44.159148 22165 raft_consensus.cc:2781] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:44.159292 22165 raft_consensus.cc:686] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [term 1 LEADER]: Becoming Leader. State: Replica: 9079ab3a642f4673958e12806a0f49a5, State: Running, Role: LEADER
I20260808 23:03:44.159371 22165 consensus_queue.cc:237] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9079ab3a642f4673958e12806a0f49a5" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44115 } } peers { permanent_uuid: "83beec735009427998fe1dd39c3cf3c3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 42105 } } peers { permanent_uuid: "cba5be0417d84e41be4fff71f15cb255" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36715 } }
I20260808 23:03:44.160211 21461 catalog_manager.cc:5495] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9079ab3a642f4673958e12806a0f49a5 (127.23.194.2). New cstate: current_term: 1 leader_uuid: "9079ab3a642f4673958e12806a0f49a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9079ab3a642f4673958e12806a0f49a5" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 44115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83beec735009427998fe1dd39c3cf3c3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 42105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cba5be0417d84e41be4fff71f15cb255" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36715 } health_report { overall_health: UNKNOWN } } }
I20260808 23:03:44.167407 22157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:44.165570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41320 (local address 127.23.194.2:44115)
0808 23:03:44.165604 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:44.165606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:44.165615 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:03:44.165685 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:44.165686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:44.165699 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:44.165754 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:03:44.165836 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.166256 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.166902 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.167063 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.167102 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:44.167196 (+ 94us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:03:44.167279 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:03:44.167334 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:44.167347 (+ 13us) server_negotiation.cc:299] Negotiation successful
0808 23:03:44.167361 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 23:03:44.168924 22072 raft_consensus.cc:1261] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3 [term 1 FOLLOWER]: Refusing update from remote peer 9079ab3a642f4673958e12806a0f49a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:03:44.168928 21680 raft_consensus.cc:1261] T 0a3a5b242e6f49008154b206f95fbe0b P cba5be0417d84e41be4fff71f15cb255 [term 1 FOLLOWER]: Refusing update from remote peer 9079ab3a642f4673958e12806a0f49a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:03:44.169246 22170 consensus_queue.cc:1035] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cba5be0417d84e41be4fff71f15cb255" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 36715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:03:44.169344 22165 consensus_queue.cc:1035] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83beec735009427998fe1dd39c3cf3c3" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 42105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:03:44.172475 22176 mvcc.cc:205] Tried to move back new op lower bound from 7316398998193696768 to 7316398998157262848. Current Snapshot: MvccSnapshot[applied={T|T < 7316398998193696768}]
I20260808 23:03:44.173846 21460 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:03:44.173939 21460 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:03:44.174674 21460 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ccc7af19488c4cbe90d0585368e3f468: Sending DeleteTablet for 3 replicas of tablet 0a3a5b242e6f49008154b206f95fbe0b
I20260808 23:03:44.175032 22042 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a3a5b242e6f49008154b206f95fbe0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:03:44 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:46106
I20260808 23:03:44.175033 21646 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a3a5b242e6f49008154b206f95fbe0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:03:44 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:33748
I20260808 23:03:44.175276 22184 tablet_replica.cc:330] T 0a3a5b242e6f49008154b206f95fbe0b P cba5be0417d84e41be4fff71f15cb255: stopping tablet replica
I20260808 23:03:44.175268 21849 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a3a5b242e6f49008154b206f95fbe0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:03:44 UTC) from {username='kudu', principal='kudu/127.23.194.62@KRBTEST.COM'} at 127.0.0.1:41314
I20260808 23:03:44.175320 22183 tablet_replica.cc:330] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3: stopping tablet replica
I20260808 23:03:44.175416 22183 raft_consensus.cc:2229] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:03:44.175556 22185 tablet_replica.cc:330] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5: stopping tablet replica
I20260808 23:03:44.175557 22183 raft_consensus.cc:2258] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:03:44.175675 22185 raft_consensus.cc:2229] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:03:44.175761 22185 raft_consensus.cc:2258] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:03:44.175835 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 21549
I20260808 23:03:44.176122 22183 ts_tablet_manager.cc:1905] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:03:44.179558 22183 ts_tablet_manager.cc:1918] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:03:44.179646 22183 log.cc:1196] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/0a3a5b242e6f49008154b206f95fbe0b
I20260808 23:03:44.179840 22183 ts_tablet_manager.cc:1939] T 0a3a5b242e6f49008154b206f95fbe0b P 83beec735009427998fe1dd39c3cf3c3: Deleting consensus metadata
I20260808 23:03:44.180145 21441 catalog_manager.cc:4841] TS 83beec735009427998fe1dd39c3cf3c3 (127.23.194.3:42105): tablet 0a3a5b242e6f49008154b206f95fbe0b (table test-table [id=ff19cba1eebc4b6a9ed33ec0cd3957af]) successfully deleted
I20260808 23:03:44.190356 22185 ts_tablet_manager.cc:1905] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:03:44.201074 22185 ts_tablet_manager.cc:1918] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:03:44.201170 22185 log.cc:1196] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/0a3a5b242e6f49008154b206f95fbe0b
I20260808 23:03:44.201470 22185 ts_tablet_manager.cc:1939] T 0a3a5b242e6f49008154b206f95fbe0b P 9079ab3a642f4673958e12806a0f49a5: Deleting consensus metadata
I20260808 23:03:44.201930 21437 catalog_manager.cc:4841] TS 9079ab3a642f4673958e12806a0f49a5 (127.23.194.2:44115): tablet 0a3a5b242e6f49008154b206f95fbe0b (table test-table [id=ff19cba1eebc4b6a9ed33ec0cd3957af]) successfully deleted
W20260808 23:03:44.204725 21441 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.194.1:36715 (error 108)
I20260808 23:03:44.204953 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 21757
I20260808 23:03:44.205185 22155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:44.205067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:36715 (local address 127.0.0.1:33760)
0808 23:03:44.205119 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:44.205151 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.1:36715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 23:03:44.205288 21441 catalog_manager.cc:4584] TS cba5be0417d84e41be4fff71f15cb255 (127.23.194.1:36715): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a3a5b242e6f49008154b206f95fbe0b: Network error: Client connection negotiation failed: client connection to 127.23.194.1:36715: connect: Connection refused (error 111)
I20260808 23:03:44.219576 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 21940
I20260808 23:03:44.228700 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 21417
2026-08-08T23:03:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2136 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T23:03:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:03:44Z Disabled control of system clock
I20260808 23:03:44.258589 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:39875
--webserver_interface=127.23.194.62
--webserver_port=0
--builtin_ntp_servers=127.23.194.20:45017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:39875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:03:44.337006 22193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:44.337212 22193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:44.337231 22193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:44.338322 22193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:03:44.338356 22193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:44.338367 22193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:44.338378 22193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:03:44.338392 22193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:03:44.339676 22193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:45017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:39875
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:39875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:44.339869 22193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:44.340099 22193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:44.344836 22203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:44.344946 22193 server_base.cc:1034] running on GCE node
W20260808 23:03:44.344852 22205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:44.344846 22202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:44.345643 22193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:44.345886 22193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:44.347016 22193 hybrid_clock.cc:648] HybridClock initialized: now 1786230224347004 us; error 28 us; skew 500 ppm
I20260808 23:03:44.348209 22193 webserver.cc:460] Webserver started at http://127.23.194.62:44115/ using document root <none> and password file <none>
I20260808 23:03:44.348433 22193 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:44.348479 22193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:44.348601 22193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:44.349647 22193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "fccd74346b3241ca9a8c80c90af7bc53"
format_stamp: "Formatted at 2026-08-08 23:03:44 on dist-test-slave-f0tn"
server_key: "81583baf647387c7a14cac1ffa9c401f"
server_key_iv: "a8cc7eaac75ea488b586970b9e984b36"
server_key_version: "encryptionkey@0"
I20260808 23:03:44.349967 22193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "fccd74346b3241ca9a8c80c90af7bc53"
format_stamp: "Formatted at 2026-08-08 23:03:44 on dist-test-slave-f0tn"
server_key: "81583baf647387c7a14cac1ffa9c401f"
server_key_iv: "a8cc7eaac75ea488b586970b9e984b36"
server_key_version: "encryptionkey@0"
I20260808 23:03:44.351190 22193 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:03:44.351845 22216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:44.352016 22193 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:44.352102 22193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "fccd74346b3241ca9a8c80c90af7bc53"
format_stamp: "Formatted at 2026-08-08 23:03:44 on dist-test-slave-f0tn"
server_key: "81583baf647387c7a14cac1ffa9c401f"
server_key_iv: "a8cc7eaac75ea488b586970b9e984b36"
server_key_version: "encryptionkey@0"
I20260808 23:03:44.352161 22193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:44.375403 22193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:44.375670 22193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:44.375771 22193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:44.379417 22193 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:39875
I20260808 23:03:44.379457 22297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:39875 every 8 connection(s)
I20260808 23:03:44.379755 22193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:03:44.380702 22298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:44.382355 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 22193
I20260808 23:03:44.382449 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:03:44.382615 24328 external_mini_cluster.cc:1428] Setting key ab7211854e59aded8b668635d0b66a35
I20260808 23:03:44.383096 22298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53: Bootstrap starting.
I20260808 23:03:44.383790 22298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:44.384115 22298 log.cc:824] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:44.384838 22298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53: No bootstrap required, opened a new log
I20260808 23:03:44.386991 22298 raft_consensus.cc:348] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fccd74346b3241ca9a8c80c90af7bc53" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39875 } }
I20260808 23:03:44.387154 22298 raft_consensus.cc:374] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:44.387192 22298 raft_consensus.cc:729] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fccd74346b3241ca9a8c80c90af7bc53, State: Initialized, Role: FOLLOWER
I20260808 23:03:44.387332 22298 consensus_queue.cc:260] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fccd74346b3241ca9a8c80c90af7bc53" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39875 } }
I20260808 23:03:44.387408 22298 raft_consensus.cc:388] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:03:44.387440 22298 raft_consensus.cc:482] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:03:44.387482 22298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:44.388041 22298 raft_consensus.cc:504] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fccd74346b3241ca9a8c80c90af7bc53" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39875 } }
I20260808 23:03:44.388154 22298 leader_election.cc:302] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fccd74346b3241ca9a8c80c90af7bc53; no voters:
I20260808 23:03:44.388338 22302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:44.383122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59450 (local address 127.23.194.62:39875)
0808 23:03:44.383310 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:44.383319 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:44.383350 (+ 31us) server_negotiation.cc:407] Connection header received
0808 23:03:44.383880 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:44.383891 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:44.383979 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:44.384130 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:44.384335 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.385600 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.386353 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.386973 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.387104 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:44.387700 (+ 596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:44.387703 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:44.387704 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:44.387870 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:44.387903 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:44.387913 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:44.387968 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:44.388065 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 23:03:44.388130 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260808 23:03:44.388362 22298 leader_election.cc:288] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:03:44.388393 22306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:44.388496 22306 raft_consensus.cc:686] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53 [term 1 LEADER]: Becoming Leader. State: Replica: fccd74346b3241ca9a8c80c90af7bc53, State: Running, Role: LEADER
I20260808 23:03:44.388572 22306 consensus_queue.cc:237] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fccd74346b3241ca9a8c80c90af7bc53" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39875 } }
I20260808 23:03:44.388602 22298 sys_catalog.cc:564] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:03:44.388885 22306 sys_catalog.cc:455] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fccd74346b3241ca9a8c80c90af7bc53. Latest consensus state: current_term: 1 leader_uuid: "fccd74346b3241ca9a8c80c90af7bc53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fccd74346b3241ca9a8c80c90af7bc53" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39875 } } }
I20260808 23:03:44.388957 22306 sys_catalog.cc:458] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:44.388895 22308 sys_catalog.cc:455] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fccd74346b3241ca9a8c80c90af7bc53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fccd74346b3241ca9a8c80c90af7bc53" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39875 } } }
I20260808 23:03:44.388995 22308 sys_catalog.cc:458] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:44.389331 22313 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:03:44.389935 22313 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:03:44.391338 22313 catalog_manager.cc:1347] Generated new cluster ID: afffd73710d34e919aadbccd2e8fe5a7
I20260808 23:03:44.391392 22313 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:03:44.398828 22313 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:03:44.399317 22313 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:03:44.406474 22313 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53: Generated new TSK 0
I20260808 23:03:44.406620 22313 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:03:44.411213 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.1:0
--local_ip_for_outbound_sockets=127.23.194.1
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:39875
--builtin_ntp_servers=127.23.194.20:45017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:03:44.492280 22329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:44.492484 22329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:44.492504 22329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:44.493665 22329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:44.493705 22329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:44.493728 22329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.1
I20260808 23:03:44.495059 22329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:45017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.194.1
--webserver_port=0
--tserver_master_addrs=127.23.194.62:39875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.1
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:44.495285 22329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:44.495558 22329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:44.500383 22338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:44.500367 22339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:44.500456 22329 server_base.cc:1034] running on GCE node
W20260808 23:03:44.500356 22343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:44.500793 22329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:44.501030 22329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:44.502180 22329 hybrid_clock.cc:648] HybridClock initialized: now 1786230224502171 us; error 28 us; skew 500 ppm
I20260808 23:03:44.503392 22329 webserver.cc:460] Webserver started at http://127.23.194.1:39595/ using document root <none> and password file <none>
I20260808 23:03:44.503620 22329 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:44.503665 22329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:44.503793 22329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:44.504808 22329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-0/data/instance:
uuid: "9c316f45ff6e4150a0dfd369c1027fa0"
format_stamp: "Formatted at 2026-08-08 23:03:44 on dist-test-slave-f0tn"
server_key: "9c52e27f48c70fdc293a79ae42da3376"
server_key_iv: "f7de67b055283f3b586760739c2c4ebd"
server_key_version: "encryptionkey@0"
I20260808 23:03:44.505133 22329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance:
uuid: "9c316f45ff6e4150a0dfd369c1027fa0"
format_stamp: "Formatted at 2026-08-08 23:03:44 on dist-test-slave-f0tn"
server_key: "9c52e27f48c70fdc293a79ae42da3376"
server_key_iv: "f7de67b055283f3b586760739c2c4ebd"
server_key_version: "encryptionkey@0"
I20260808 23:03:44.506441 22329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:03:44.507148 22351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:44.507304 22329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:44.507400 22329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-0/wal
uuid: "9c316f45ff6e4150a0dfd369c1027fa0"
format_stamp: "Formatted at 2026-08-08 23:03:44 on dist-test-slave-f0tn"
server_key: "9c52e27f48c70fdc293a79ae42da3376"
server_key_iv: "f7de67b055283f3b586760739c2c4ebd"
server_key_version: "encryptionkey@0"
I20260808 23:03:44.507463 22329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:44.522550 22329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:44.522790 22329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:44.522886 22329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:44.523049 22329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:44.523363 22329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:44.523399 22329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:44.523427 22329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:44.523444 22329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:44.528779 22329 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.1:34805
I20260808 23:03:44.528838 22513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.1:34805 every 8 connection(s)
I20260808 23:03:44.529114 22329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-0/data/info.pb
I20260808 23:03:44.533571 22302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:44.529803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:44231 (local address 127.23.194.62:39875)
0808 23:03:44.529854 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:44.529856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:44.530051 (+ 195us) server_negotiation.cc:407] Connection header received
0808 23:03:44.530496 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:44.530498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:44.530506 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:44.530561 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:44.531161 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.531632 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.532482 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.532654 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.532694 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:44.533326 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:44.533328 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:44.533328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:44.533332 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:44.533346 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:44.533350 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:44.533431 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:44.533490 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 23:03:44.533512 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 23:03:44.533756 22519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:44.529771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:39875 (local address 127.23.194.1:44231)
0808 23:03:44.529942 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:44.529980 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:44.530425 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:44.530612 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:44.530615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:44.530622 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:44.531091 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:44.531095 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.531689 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.531690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:44.532425 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:44.532427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.532704 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.532705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:44.532802 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:44.533214 (+ 412us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:03:44.533214 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:44.533239 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:03:44.533259 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:44.533434 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:44.533437 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:44.533439 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 23:03:44.533486 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 23:03:44.533558 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260808 23:03:44.534161 22517 heartbeater.cc:347] Connected to a master server at 127.23.194.62:39875
I20260808 23:03:44.534240 22517 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:44.534399 22517 heartbeater.cc:511] Master 127.23.194.62:39875 requested a full tablet report, sending...
I20260808 23:03:44.534828 22249 ts_manager.cc:194] Registered new tserver with Master: 9c316f45ff6e4150a0dfd369c1027fa0 (127.23.194.1:34805)
I20260808 23:03:44.535424 22249 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.194.1:44231
I20260808 23:03:44.536458 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 22329
I20260808 23:03:44.536549 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-0/wal/instance
I20260808 23:03:44.536693 24328 external_mini_cluster.cc:1428] Setting key b678c85562ed25f60310538468f0195c
I20260808 23:03:44.537492 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.2:0
--local_ip_for_outbound_sockets=127.23.194.2
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:39875
--builtin_ntp_servers=127.23.194.20:45017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:03:44.616822 22522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:44.617009 22522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:44.617038 22522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:44.618160 22522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:44.618199 22522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:44.618232 22522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.2
I20260808 23:03:44.619557 22522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:45017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.194.2
--webserver_port=0
--tserver_master_addrs=127.23.194.62:39875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.2
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:44.619793 22522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:44.620064 22522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:44.624804 22532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:44.624776 22538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:44.624788 22530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:44.624823 22522 server_base.cc:1034] running on GCE node
I20260808 23:03:44.625137 22522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:44.625365 22522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:44.626538 22522 hybrid_clock.cc:648] HybridClock initialized: now 1786230224626517 us; error 37 us; skew 500 ppm
I20260808 23:03:44.627750 22522 webserver.cc:460] Webserver started at http://127.23.194.2:46615/ using document root <none> and password file <none>
I20260808 23:03:44.627985 22522 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:44.628031 22522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:44.628165 22522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:44.629160 22522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-1/data/instance:
uuid: "146942931a5640af80f061ea309d7b25"
format_stamp: "Formatted at 2026-08-08 23:03:44 on dist-test-slave-f0tn"
server_key: "e8b3d743191dde55761ca851226a5cf7"
server_key_iv: "42a52f124c3fc2bdf2557f14c7af1aca"
server_key_version: "encryptionkey@0"
I20260808 23:03:44.629493 22522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance:
uuid: "146942931a5640af80f061ea309d7b25"
format_stamp: "Formatted at 2026-08-08 23:03:44 on dist-test-slave-f0tn"
server_key: "e8b3d743191dde55761ca851226a5cf7"
server_key_iv: "42a52f124c3fc2bdf2557f14c7af1aca"
server_key_version: "encryptionkey@0"
I20260808 23:03:44.630642 22522 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 23:03:44.631271 22548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:44.631417 22522 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:03:44.631493 22522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-1/wal
uuid: "146942931a5640af80f061ea309d7b25"
format_stamp: "Formatted at 2026-08-08 23:03:44 on dist-test-slave-f0tn"
server_key: "e8b3d743191dde55761ca851226a5cf7"
server_key_iv: "42a52f124c3fc2bdf2557f14c7af1aca"
server_key_version: "encryptionkey@0"
I20260808 23:03:44.631556 22522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:44.646978 22522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:44.647233 22522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:44.647353 22522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:44.647545 22522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:44.647845 22522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:44.647877 22522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:44.647908 22522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:44.647933 22522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:44.653277 22522 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.2:41917
I20260808 23:03:44.653337 22721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.2:41917 every 8 connection(s)
I20260808 23:03:44.653625 22522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-1/data/info.pb
I20260808 23:03:44.658205 22302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:44.654255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.2:57337 (local address 127.23.194.62:39875)
0808 23:03:44.654304 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:44.654306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:44.654513 (+ 207us) server_negotiation.cc:407] Connection header received
0808 23:03:44.655028 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:44.655028 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:44.655039 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:44.655089 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:44.655719 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.656221 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.657119 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.657303 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.657343 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:44.658010 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:44.658011 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:44.658012 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:44.658015 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:44.658030 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:44.658034 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:44.658073 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:44.658117 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 23:03:44.658139 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 23:03:44.658380 22729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:44.654236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:39875 (local address 127.23.194.2:57337)
0808 23:03:44.654399 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:44.654435 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:44.654950 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:44.655142 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:44.655145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:44.655153 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:44.655653 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:44.655658 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.656288 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.656289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:44.657060 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:44.657062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.657355 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.657356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:44.657474 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:44.657893 (+ 419us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:03:44.657893 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:44.657919 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:03:44.657940 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:44.658073 (+ 133us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:44.658075 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:44.658078 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 23:03:44.658118 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 23:03:44.658188 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":83,"threads_started":1}
I20260808 23:03:44.658761 22723 heartbeater.cc:347] Connected to a master server at 127.23.194.62:39875
I20260808 23:03:44.658841 22723 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:44.659000 22723 heartbeater.cc:511] Master 127.23.194.62:39875 requested a full tablet report, sending...
I20260808 23:03:44.659322 22249 ts_manager.cc:194] Registered new tserver with Master: 146942931a5640af80f061ea309d7b25 (127.23.194.2:41917)
I20260808 23:03:44.659667 22249 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.194.2:57337
I20260808 23:03:44.662277 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 22522
I20260808 23:03:44.662366 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-1/wal/instance
I20260808 23:03:44.662505 24328 external_mini_cluster.cc:1428] Setting key c299fd693337f47f5c36827b084076dd
I20260808 23:03:44.663298 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.194.3:0
--local_ip_for_outbound_sockets=127.23.194.3
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:39875
--builtin_ntp_servers=127.23.194.20:45017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:03:44.742620 22730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:44.742803 22730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:44.742823 22730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:44.743973 22730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:44.744012 22730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:44.744037 22730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.194.3
I20260808 23:03:44.745371 22730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:45017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.194.3
--webserver_port=0
--tserver_master_addrs=127.23.194.62:39875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.194.3
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:44.745615 22730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:44.745882 22730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:44.750530 22740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:44.750609 22730 server_base.cc:1034] running on GCE node
W20260808 23:03:44.750572 22738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:44.750567 22736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:44.750910 22730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:44.751132 22730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:44.752274 22730 hybrid_clock.cc:648] HybridClock initialized: now 1786230224752261 us; error 30 us; skew 500 ppm
I20260808 23:03:44.753417 22730 webserver.cc:460] Webserver started at http://127.23.194.3:43797/ using document root <none> and password file <none>
I20260808 23:03:44.753655 22730 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:44.753700 22730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:44.753823 22730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:44.754885 22730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-2/data/instance:
uuid: "2cd5189282d642d397f920275105780b"
format_stamp: "Formatted at 2026-08-08 23:03:44 on dist-test-slave-f0tn"
server_key: "74ad281a6510fa2681456bfde2e6ddae"
server_key_iv: "3654168e31dcc5374f098aae2bcd4d69"
server_key_version: "encryptionkey@0"
I20260808 23:03:44.755203 22730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance:
uuid: "2cd5189282d642d397f920275105780b"
format_stamp: "Formatted at 2026-08-08 23:03:44 on dist-test-slave-f0tn"
server_key: "74ad281a6510fa2681456bfde2e6ddae"
server_key_iv: "3654168e31dcc5374f098aae2bcd4d69"
server_key_version: "encryptionkey@0"
I20260808 23:03:44.756364 22730 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:03:44.756980 22757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:44.757113 22730 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:44.757196 22730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-2/wal
uuid: "2cd5189282d642d397f920275105780b"
format_stamp: "Formatted at 2026-08-08 23:03:44 on dist-test-slave-f0tn"
server_key: "74ad281a6510fa2681456bfde2e6ddae"
server_key_iv: "3654168e31dcc5374f098aae2bcd4d69"
server_key_version: "encryptionkey@0"
I20260808 23:03:44.757261 22730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:44.771962 22730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:44.772248 22730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:44.772372 22730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:44.772575 22730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:03:44.772894 22730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:03:44.772928 22730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:44.772955 22730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:03:44.772979 22730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:44.778513 22730 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.3:33223
I20260808 23:03:44.778582 22937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.3:33223 every 8 connection(s)
I20260808 23:03:44.778872 22730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-2/data/info.pb
I20260808 23:03:44.783497 22302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:44.779537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.3:44729 (local address 127.23.194.62:39875)
0808 23:03:44.779589 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:44.779591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:44.779844 (+ 253us) server_negotiation.cc:407] Connection header received
0808 23:03:44.780297 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:44.780298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:44.780309 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:44.780360 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:44.780969 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.781498 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.782413 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.782588 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.782630 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:44.783293 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:44.783294 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:44.783294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:44.783298 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:44.783313 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:44.783316 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:44.783355 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:44.783416 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 23:03:44.783439 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 23:03:44.783680 22940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:44.779537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:39875 (local address 127.23.194.3:44729)
0808 23:03:44.779728 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:44.779762 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:44.780221 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:44.780420 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:44.780423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:44.780431 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:03:44.780909 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:44.780913 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.781567 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.781568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:44.782352 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:44.782355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.782638 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.782639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:44.782738 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:44.783171 (+ 433us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:03:44.783172 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:03:44.783198 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:03:44.783220 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:03:44.783360 (+ 140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:03:44.783363 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:03:44.783365 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 23:03:44.783409 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 23:03:44.783484 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260808 23:03:44.784144 22938 heartbeater.cc:347] Connected to a master server at 127.23.194.62:39875
I20260808 23:03:44.784238 22938 heartbeater.cc:464] Registering TS with master...
I20260808 23:03:44.784381 22938 heartbeater.cc:511] Master 127.23.194.62:39875 requested a full tablet report, sending...
I20260808 23:03:44.784713 22249 ts_manager.cc:194] Registered new tserver with Master: 2cd5189282d642d397f920275105780b (127.23.194.3:33223)
I20260808 23:03:44.785055 22249 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.194.3:44729
I20260808 23:03:44.788194 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 22730
I20260808 23:03:44.788300 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-2/wal/instance
I20260808 23:03:44.788494 24328 external_mini_cluster.cc:1428] Setting key 5e8702304f3ad00cab6f41d7c8ccf784
I20260808 23:03:44.789256 24328 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:03:44.792918 22302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:44.790593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52282 (local address 127.23.194.62:39875)
0808 23:03:44.790626 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:44.790628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:44.790723 (+ 95us) server_negotiation.cc:407] Connection header received
0808 23:03:44.790773 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:44.790774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:44.790781 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:44.790825 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:44.790933 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.791370 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.792020 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.792186 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.792226 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:44.792768 (+ 542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:44.792768 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:44.792769 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:44.792772 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:44.792783 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:44.792786 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:44.792820 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:44.792853 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 23:03:44.792870 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 23:03:44.794332 22249 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:03:44.795796 22249 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:03:44.801482 22962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:44.797686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:41917 (local address 127.0.0.1:59540)
0808 23:03:44.797952 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:44.798001 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:44.798063 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:44.798293 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:44.798295 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:44.798296 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:44.798341 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:44.798343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.799602 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.799603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:44.800572 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:44.800574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.801188 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.801189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:44.801266 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:44.801279 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 23:03:44.801340 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 23:03:44.801416 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":99,"threads_started":1}
I20260808 23:03:44.801494 22963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:44.797783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59540 (local address 127.23.194.2:41917)
0808 23:03:44.798165 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:44.798169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:44.798179 (+ 10us) server_negotiation.cc:407] Connection header received
0808 23:03:44.798212 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:44.798214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:44.798226 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:44.798293 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:44.798428 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.799508 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.800632 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.801107 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.801179 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:44.801269 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:44.801352 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 23:03:44.801411 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":84,"threads_started":1}
I20260808 23:03:44.802243 22954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:44.797692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:34805 (local address 127.0.0.1:58204)
0808 23:03:44.798676 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:44.798685 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:44.798734 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:44.798891 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:44.798891 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:44.798893 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:44.798928 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:44.798929 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.800119 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.800120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:44.801164 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:44.801167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.802083 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.802084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:44.802130 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:44.802134 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:03:44.802172 (+ 38us) client_negotiation.cc:240] Negotiation successful
0808 23:03:44.802198 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"mutex_wait_us":7,"thread_start_us":97,"threads_started":1}
I20260808 23:03:44.802299 22966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:44.797935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58204 (local address 127.23.194.1:34805)
0808 23:03:44.798093 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:44.798097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:44.798736 (+ 639us) server_negotiation.cc:407] Connection header received
0808 23:03:44.798794 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:44.798797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:44.798812 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:44.798885 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:44.798983 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.800038 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.801261 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.802003 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.802085 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:44.802165 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:44.802189 (+ 24us) server_negotiation.cc:299] Negotiation successful
0808 23:03:44.802224 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260808 23:03:44.802263 22626 tablet_service.cc:1467] Processing CreateTablet for tablet 49398473bab74dff92d0352171c218c1 (DEFAULT_TABLE table=test-table [id=6b351930006a4e0e801bb6e74d84d90a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:44.802627 22626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49398473bab74dff92d0352171c218c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:44.805058 22967 tablet_bootstrap.cc:492] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25: Bootstrap starting.
I20260808 23:03:44.805358 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:44.797874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35380 (local address 127.23.194.3:33223)
0808 23:03:44.798022 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:44.798026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:44.798477 (+ 451us) server_negotiation.cc:407] Connection header received
0808 23:03:44.798516 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:44.798518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:44.798532 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:44.798597 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:44.798720 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.799448 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.800535 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.801226 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.801278 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:44.801358 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:44.801808 (+ 450us) server_negotiation.cc:299] Negotiation successful
0808 23:03:44.805282 (+ 3474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260808 23:03:44.805584 22952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:44.797688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:33223 (local address 127.0.0.1:35380)
0808 23:03:44.798365 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:44.798375 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:44.798423 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:44.798592 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:44.798593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:44.798594 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:44.798641 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:44.798643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.799515 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.799516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:44.800444 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:44.800446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:44.801412 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:44.801413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:44.801467 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:44.801471 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:03:44.801517 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 23:03:44.805522 (+ 4005us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":48,"threads_started":1}
I20260808 23:03:44.805680 22422 tablet_service.cc:1467] Processing CreateTablet for tablet 49398473bab74dff92d0352171c218c1 (DEFAULT_TABLE table=test-table [id=6b351930006a4e0e801bb6e74d84d90a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:44.805997 22967 tablet_bootstrap.cc:654] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:44.806042 22422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49398473bab74dff92d0352171c218c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:44.806327 22967 log.cc:824] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:44.806272 22840 tablet_service.cc:1467] Processing CreateTablet for tablet 49398473bab74dff92d0352171c218c1 (DEFAULT_TABLE table=test-table [id=6b351930006a4e0e801bb6e74d84d90a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:03:44.806587 22840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49398473bab74dff92d0352171c218c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:44.807077 22967 tablet_bootstrap.cc:492] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25: No bootstrap required, opened a new log
I20260808 23:03:44.807150 22967 ts_tablet_manager.cc:1397] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 23:03:44.808529 22969 tablet_bootstrap.cc:492] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0: Bootstrap starting.
I20260808 23:03:44.809093 22969 tablet_bootstrap.cc:654] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:44.809568 22969 log.cc:824] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:44.809648 22967 raft_consensus.cc:348] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146942931a5640af80f061ea309d7b25" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41917 } } peers { permanent_uuid: "9c316f45ff6e4150a0dfd369c1027fa0" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34805 } } peers { permanent_uuid: "2cd5189282d642d397f920275105780b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33223 } }
I20260808 23:03:44.809809 22967 raft_consensus.cc:374] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:44.809846 22967 raft_consensus.cc:729] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 146942931a5640af80f061ea309d7b25, State: Initialized, Role: FOLLOWER
I20260808 23:03:44.809911 22970 tablet_bootstrap.cc:492] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b: Bootstrap starting.
I20260808 23:03:44.809954 22967 consensus_queue.cc:260] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146942931a5640af80f061ea309d7b25" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41917 } } peers { permanent_uuid: "9c316f45ff6e4150a0dfd369c1027fa0" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34805 } } peers { permanent_uuid: "2cd5189282d642d397f920275105780b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33223 } }
I20260808 23:03:44.810226 22967 ts_tablet_manager.cc:1428] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 23:03:44.810268 22723 heartbeater.cc:503] Master 127.23.194.62:39875 was elected leader, sending a full tablet report...
I20260808 23:03:44.810498 22970 tablet_bootstrap.cc:654] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:44.810803 22970 log.cc:824] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:44.811499 22970 tablet_bootstrap.cc:492] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b: No bootstrap required, opened a new log
I20260808 23:03:44.811553 22970 ts_tablet_manager.cc:1397] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:03:44.811573 22969 tablet_bootstrap.cc:492] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0: No bootstrap required, opened a new log
I20260808 23:03:44.811642 22969 ts_tablet_manager.cc:1397] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 23:03:44.813252 22970 raft_consensus.cc:348] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146942931a5640af80f061ea309d7b25" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41917 } } peers { permanent_uuid: "9c316f45ff6e4150a0dfd369c1027fa0" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34805 } } peers { permanent_uuid: "2cd5189282d642d397f920275105780b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33223 } }
I20260808 23:03:44.813398 22970 raft_consensus.cc:374] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:44.813437 22970 raft_consensus.cc:729] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd5189282d642d397f920275105780b, State: Initialized, Role: FOLLOWER
I20260808 23:03:44.813421 22969 raft_consensus.cc:348] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146942931a5640af80f061ea309d7b25" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41917 } } peers { permanent_uuid: "9c316f45ff6e4150a0dfd369c1027fa0" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34805 } } peers { permanent_uuid: "2cd5189282d642d397f920275105780b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33223 } }
I20260808 23:03:44.813575 22969 raft_consensus.cc:374] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:44.813550 22970 consensus_queue.cc:260] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146942931a5640af80f061ea309d7b25" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41917 } } peers { permanent_uuid: "9c316f45ff6e4150a0dfd369c1027fa0" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34805 } } peers { permanent_uuid: "2cd5189282d642d397f920275105780b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33223 } }
I20260808 23:03:44.813616 22969 raft_consensus.cc:729] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c316f45ff6e4150a0dfd369c1027fa0, State: Initialized, Role: FOLLOWER
I20260808 23:03:44.813724 22969 consensus_queue.cc:260] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146942931a5640af80f061ea309d7b25" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41917 } } peers { permanent_uuid: "9c316f45ff6e4150a0dfd369c1027fa0" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34805 } } peers { permanent_uuid: "2cd5189282d642d397f920275105780b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33223 } }
I20260808 23:03:44.813813 22970 ts_tablet_manager.cc:1428] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 23:03:44.813840 22938 heartbeater.cc:503] Master 127.23.194.62:39875 was elected leader, sending a full tablet report...
I20260808 23:03:44.813968 22969 ts_tablet_manager.cc:1428] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 23:03:44.814038 22517 heartbeater.cc:503] Master 127.23.194.62:39875 was elected leader, sending a full tablet report...
W20260808 23:03:44.904134 22724 tablet.cc:2369] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:03:45.029417 22939 tablet.cc:2369] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:03:45.029716 22518 tablet.cc:2369] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:03:45.142633 22977 raft_consensus.cc:482] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:03:45.142764 22977 raft_consensus.cc:504] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146942931a5640af80f061ea309d7b25" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41917 } } peers { permanent_uuid: "9c316f45ff6e4150a0dfd369c1027fa0" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34805 } } peers { permanent_uuid: "2cd5189282d642d397f920275105780b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33223 } }
I20260808 23:03:45.143191 22977 leader_election.cc:288] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 146942931a5640af80f061ea309d7b25 (127.23.194.2:41917), 2cd5189282d642d397f920275105780b (127.23.194.3:33223)
I20260808 23:03:45.146476 22963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:45.143393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:43495 (local address 127.23.194.2:41917)
0808 23:03:45.143436 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:45.143438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:45.143560 (+ 122us) server_negotiation.cc:407] Connection header received
0808 23:03:45.143677 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:45.143679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:45.143689 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:45.143754 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:45.143968 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:45.144574 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:45.145577 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:45.146065 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:45.146115 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:45.146124 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:45.146352 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 23:03:45.146398 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 23:03:45.146749 22981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:45.143340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.2:41917 (local address 127.23.194.1:43495)
0808 23:03:45.143485 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:45.143494 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:45.143593 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:45.143805 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:45.143806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:45.143808 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:45.143888 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:45.143890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:45.144649 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:45.144650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:45.145488 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:45.145491 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:45.146195 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:45.146196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:45.146267 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:45.146271 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:03:45.146608 (+ 337us) client_negotiation.cc:240] Negotiation successful
0808 23:03:45.146655 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":52,"threads_started":1}
I20260808 23:03:45.147082 22663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49398473bab74dff92d0352171c218c1" candidate_uuid: "9c316f45ff6e4150a0dfd369c1027fa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146942931a5640af80f061ea309d7b25" is_pre_election: true
I20260808 23:03:45.147258 22663 raft_consensus.cc:2454] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c316f45ff6e4150a0dfd369c1027fa0 in term 0.
I20260808 23:03:45.147492 22357 leader_election.cc:302] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 146942931a5640af80f061ea309d7b25, 9c316f45ff6e4150a0dfd369c1027fa0; no voters:
I20260808 23:03:45.147756 22977 raft_consensus.cc:2781] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:03:45.147853 22977 raft_consensus.cc:482] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:03:45.147898 22977 raft_consensus.cc:3037] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:45.148469 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:45.143574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.194.1:59579 (local address 127.23.194.3:33223)
0808 23:03:45.143618 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:45.143620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:45.143696 (+ 76us) server_negotiation.cc:407] Connection header received
0808 23:03:45.143775 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:45.143776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:45.143785 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:45.143838 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:03:45.145834 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:45.146335 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:45.147617 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:45.148101 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:45.148172 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:45.148179 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:45.148362 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 23:03:45.148402 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 23:03:45.148677 22978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:45.143385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.3:33223 (local address 127.23.194.1:59579)
0808 23:03:45.143622 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:45.143632 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:03:45.143711 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:03:45.145675 (+ 1964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:03:45.145676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:03:45.145679 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:03:45.145751 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:45.145753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:45.146408 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:45.146409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:45.147544 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:03:45.147547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:03:45.148212 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:03:45.148213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:03:45.148283 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:45.148287 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:03:45.148553 (+ 266us) client_negotiation.cc:240] Negotiation successful
0808 23:03:45.148601 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":59,"threads_started":1}
I20260808 23:03:45.148952 22977 raft_consensus.cc:504] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146942931a5640af80f061ea309d7b25" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41917 } } peers { permanent_uuid: "9c316f45ff6e4150a0dfd369c1027fa0" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34805 } } peers { permanent_uuid: "2cd5189282d642d397f920275105780b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33223 } }
I20260808 23:03:45.149093 22977 leader_election.cc:288] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0 [CANDIDATE]: Term 1 election: Requested vote from peers 146942931a5640af80f061ea309d7b25 (127.23.194.2:41917), 2cd5189282d642d397f920275105780b (127.23.194.3:33223)
I20260808 23:03:45.149057 22872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49398473bab74dff92d0352171c218c1" candidate_uuid: "9c316f45ff6e4150a0dfd369c1027fa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd5189282d642d397f920275105780b" is_pre_election: true
I20260808 23:03:45.149195 22872 raft_consensus.cc:2454] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c316f45ff6e4150a0dfd369c1027fa0 in term 0.
I20260808 23:03:45.149231 22663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49398473bab74dff92d0352171c218c1" candidate_uuid: "9c316f45ff6e4150a0dfd369c1027fa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146942931a5640af80f061ea309d7b25"
I20260808 23:03:45.149323 22663 raft_consensus.cc:3037] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:45.149469 22872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49398473bab74dff92d0352171c218c1" candidate_uuid: "9c316f45ff6e4150a0dfd369c1027fa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd5189282d642d397f920275105780b"
I20260808 23:03:45.149576 22872 raft_consensus.cc:3037] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:45.150229 22663 raft_consensus.cc:2454] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c316f45ff6e4150a0dfd369c1027fa0 in term 1.
I20260808 23:03:45.150430 22872 raft_consensus.cc:2454] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c316f45ff6e4150a0dfd369c1027fa0 in term 1.
I20260808 23:03:45.150599 22352 leader_election.cc:302] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd5189282d642d397f920275105780b, 9c316f45ff6e4150a0dfd369c1027fa0; no voters:
I20260808 23:03:45.150740 22977 raft_consensus.cc:2781] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:45.150902 22977 raft_consensus.cc:686] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0 [term 1 LEADER]: Becoming Leader. State: Replica: 9c316f45ff6e4150a0dfd369c1027fa0, State: Running, Role: LEADER
I20260808 23:03:45.151005 22977 consensus_queue.cc:237] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146942931a5640af80f061ea309d7b25" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41917 } } peers { permanent_uuid: "9c316f45ff6e4150a0dfd369c1027fa0" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34805 } } peers { permanent_uuid: "2cd5189282d642d397f920275105780b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33223 } }
I20260808 23:03:45.151837 22249 catalog_manager.cc:5495] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c316f45ff6e4150a0dfd369c1027fa0 (127.23.194.1). New cstate: current_term: 1 leader_uuid: "9c316f45ff6e4150a0dfd369c1027fa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146942931a5640af80f061ea309d7b25" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c316f45ff6e4150a0dfd369c1027fa0" member_type: VOTER last_known_addr { host: "127.23.194.1" port: 34805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cd5189282d642d397f920275105780b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33223 } health_report { overall_health: UNKNOWN } } }
I20260808 23:03:45.239015 22966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:45.237007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58218 (local address 127.23.194.1:34805)
0808 23:03:45.237064 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:45.237065 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:45.237073 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:03:45.237110 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:45.237110 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:45.237131 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:45.237200 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:03:45.237291 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:45.237763 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:45.238431 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:45.238605 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:45.238647 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:45.238758 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:03:45.238852 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:03:45.238906 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:45.238924 (+ 18us) server_negotiation.cc:299] Negotiation successful
0808 23:03:45.238939 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 23:03:45.240832 22663 raft_consensus.cc:1261] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25 [term 1 FOLLOWER]: Refusing update from remote peer 9c316f45ff6e4150a0dfd369c1027fa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:03:45.240849 22872 raft_consensus.cc:1261] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b [term 1 FOLLOWER]: Refusing update from remote peer 9c316f45ff6e4150a0dfd369c1027fa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:03:45.241186 22977 consensus_queue.cc:1035] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cd5189282d642d397f920275105780b" member_type: VOTER last_known_addr { host: "127.23.194.3" port: 33223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:03:45.241321 22982 consensus_queue.cc:1035] T 49398473bab74dff92d0352171c218c1 P 9c316f45ff6e4150a0dfd369c1027fa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "146942931a5640af80f061ea309d7b25" member_type: VOTER last_known_addr { host: "127.23.194.2" port: 41917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:03:45.243603 22988 mvcc.cc:205] Tried to move back new op lower bound from 7316399002583773184 to 7316399002219188224. Current Snapshot: MvccSnapshot[applied={T|T < 7316399002583773184}]
I20260808 23:03:45.243769 22996 mvcc.cc:205] Tried to move back new op lower bound from 7316399002583773184 to 7316399002219188224. Current Snapshot: MvccSnapshot[applied={T|T < 7316399002583773184}]
I20260808 23:03:45.244056 22991 mvcc.cc:205] Tried to move back new op lower bound from 7316399002583773184 to 7316399002219188224. Current Snapshot: MvccSnapshot[applied={T|T < 7316399002583773184}]
I20260808 23:03:45.246362 22249 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:03:45.246443 22249 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:03:45.247112 22249 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fccd74346b3241ca9a8c80c90af7bc53: Sending DeleteTablet for 3 replicas of tablet 49398473bab74dff92d0352171c218c1
I20260808 23:03:45.247408 22626 tablet_service.cc:1514] Processing DeleteTablet for tablet 49398473bab74dff92d0352171c218c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:03:45 UTC) from {username='slave'} at 127.0.0.1:59540
I20260808 23:03:45.247411 22422 tablet_service.cc:1514] Processing DeleteTablet for tablet 49398473bab74dff92d0352171c218c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:03:45 UTC) from {username='slave'} at 127.0.0.1:58204
I20260808 23:03:45.247624 23010 tablet_replica.cc:330] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25: stopping tablet replica
I20260808 23:03:45.247725 23010 raft_consensus.cc:2229] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:03:45.247757 22840 tablet_service.cc:1514] Processing DeleteTablet for tablet 49398473bab74dff92d0352171c218c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:03:45 UTC) from {username='slave'} at 127.0.0.1:35380
I20260808 23:03:45.247843 23010 raft_consensus.cc:2258] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:03:45.248083 23015 tablet_replica.cc:330] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b: stopping tablet replica
I20260808 23:03:45.248174 23015 raft_consensus.cc:2229] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:03:45.248215 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 22329
I20260808 23:03:45.248258 23015 raft_consensus.cc:2258] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:03:45.249027 23015 ts_tablet_manager.cc:1905] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:03:45.251008 23015 ts_tablet_manager.cc:1918] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:03:45.251072 23015 log.cc:1196] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-2/wal/wals/49398473bab74dff92d0352171c218c1
I20260808 23:03:45.251293 23015 ts_tablet_manager.cc:1939] T 49398473bab74dff92d0352171c218c1 P 2cd5189282d642d397f920275105780b: Deleting consensus metadata
I20260808 23:03:45.251641 22218 catalog_manager.cc:4841] TS 2cd5189282d642d397f920275105780b (127.23.194.3:33223): tablet 49398473bab74dff92d0352171c218c1 (table test-table [id=6b351930006a4e0e801bb6e74d84d90a]) successfully deleted
I20260808 23:03:45.267433 23010 ts_tablet_manager.cc:1905] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:03:45.268803 23010 ts_tablet_manager.cc:1918] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:03:45.268865 23010 log.cc:1196] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25: Deleting WAL directory at /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786229981209642-24328-0/minicluster-data/ts-1/wal/wals/49398473bab74dff92d0352171c218c1
I20260808 23:03:45.269084 23010 ts_tablet_manager.cc:1939] T 49398473bab74dff92d0352171c218c1 P 146942931a5640af80f061ea309d7b25: Deleting consensus metadata
I20260808 23:03:45.269500 22223 catalog_manager.cc:4841] TS 146942931a5640af80f061ea309d7b25 (127.23.194.2:41917): tablet 49398473bab74dff92d0352171c218c1 (table test-table [id=6b351930006a4e0e801bb6e74d84d90a]) successfully deleted
W20260808 23:03:45.271613 22219 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.194.1:34805 (error 108)
I20260808 23:03:45.272078 22952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:45.271963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.1:34805 (local address 127.0.0.1:58234)
0808 23:03:45.272010 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 23:03:45.272050 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.1:34805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 23:03:45.272115 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 22522
W20260808 23:03:45.272207 22219 catalog_manager.cc:4584] TS 9c316f45ff6e4150a0dfd369c1027fa0 (127.23.194.1:34805): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 49398473bab74dff92d0352171c218c1: Network error: Client connection negotiation failed: client connection to 127.23.194.1:34805: connect: Connection refused (error 111)
I20260808 23:03:45.281037 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 22730
I20260808 23:03:45.294930 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 22193
2026-08-08T23:03:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1067 ms)
[----------] 6 tests from EncryptionPolicyTest (8841 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:03:45 dist-test-slave-f0tn krb5kdc[23019](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:03:45 dist-test-slave-f0tn krb5kdc[23019](info): set up 2 sockets
Aug 08 23:03:45 dist-test-slave-f0tn krb5kdc[23019](info): commencing operation
krb5kdc: starting...
W20260808 23:03:46.426779 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:03:46 dist-test-slave-f0tn krb5kdc[23019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:03:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:03:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:03:46.539778 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36847
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:46.619419 23035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:46.619643 23035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:46.619668 23035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:46.620752 23035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:03:46.620785 23035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:46.620798 23035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:46.620810 23035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:03:46.620822 23035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:03:46.622251 23035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36847
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:46.622421 23035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:46.622591 23035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:46.627097 23045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:46.627163 23042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:46.627169 23035 server_base.cc:1034] running on GCE node
W20260808 23:03:46.627106 23041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:46.627525 23035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:46.627813 23035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:46.628953 23035 hybrid_clock.cc:648] HybridClock initialized: now 1786230226628936 us; error 31 us; skew 500 ppm
Aug 08 23:03:46 dist-test-slave-f0tn krb5kdc[23019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230226, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:46.635313 23035 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:03:46.635721 23035 webserver.cc:460] Webserver started at http://127.0.0.1:33611/ using document root <none> and password file <none>
I20260808 23:03:46.635936 23035 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:46.635985 23035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:46.636104 23035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:46.637107 23035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "127fd390c9744fb9ab1efc11c2394640"
format_stamp: "Formatted at 2026-08-08 23:03:46 on dist-test-slave-f0tn"
server_key: "7304991f7bfed8ec5f06bc1728515191"
server_key_iv: "2c38c1139a5b330dfb821d559b3ab498"
server_key_version: "encryptionkey@0"
I20260808 23:03:46.637477 23035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "127fd390c9744fb9ab1efc11c2394640"
format_stamp: "Formatted at 2026-08-08 23:03:46 on dist-test-slave-f0tn"
server_key: "7304991f7bfed8ec5f06bc1728515191"
server_key_iv: "2c38c1139a5b330dfb821d559b3ab498"
server_key_version: "encryptionkey@0"
I20260808 23:03:46.638697 23035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:03:46.639326 23056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:46.639472 23035 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:46.639554 23035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "127fd390c9744fb9ab1efc11c2394640"
format_stamp: "Formatted at 2026-08-08 23:03:46 on dist-test-slave-f0tn"
server_key: "7304991f7bfed8ec5f06bc1728515191"
server_key_iv: "2c38c1139a5b330dfb821d559b3ab498"
server_key_version: "encryptionkey@0"
I20260808 23:03:46.639617 23035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:46.656081 23035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:46.657732 23035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:46.657866 23035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:46.661538 23035 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36847
I20260808 23:03:46.661580 23133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36847 every 8 connection(s)
I20260808 23:03:46.661918 23035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:03:46.662716 23137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:46.663877 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 23035
I20260808 23:03:46.664011 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:03:46.664202 24328 external_mini_cluster.cc:1428] Setting key 592eb33551d4f2c6752c963d027b7bbb
I20260808 23:03:46.665266 23137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640: Bootstrap starting.
I20260808 23:03:46.665879 23137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:46.666216 23137 log.cc:824] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:46.666887 23137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640: No bootstrap required, opened a new log
I20260808 23:03:46.668980 23137 raft_consensus.cc:348] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "127fd390c9744fb9ab1efc11c2394640" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36847 } }
I20260808 23:03:46.669147 23137 raft_consensus.cc:374] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:46.669195 23137 raft_consensus.cc:729] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 127fd390c9744fb9ab1efc11c2394640, State: Initialized, Role: FOLLOWER
I20260808 23:03:46.669324 23137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "127fd390c9744fb9ab1efc11c2394640" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36847 } }
I20260808 23:03:46.669421 23137 raft_consensus.cc:388] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:03:46.669462 23137 raft_consensus.cc:482] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:03:46.669508 23137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:46.670002 23137 raft_consensus.cc:504] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "127fd390c9744fb9ab1efc11c2394640" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36847 } }
I20260808 23:03:46.670087 23137 leader_election.cc:302] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 127fd390c9744fb9ab1efc11c2394640; no voters:
I20260808 23:03:46.670244 23137 leader_election.cc:288] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:03:46.670284 23143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:46.670426 23143 raft_consensus.cc:686] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640 [term 1 LEADER]: Becoming Leader. State: Replica: 127fd390c9744fb9ab1efc11c2394640, State: Running, Role: LEADER
I20260808 23:03:46.670488 23137 sys_catalog.cc:564] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:03:46.670519 23143 consensus_queue.cc:237] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "127fd390c9744fb9ab1efc11c2394640" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36847 } }
Aug 08 23:03:46 dist-test-slave-f0tn krb5kdc[23019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 23:03:46.670832 23143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 127fd390c9744fb9ab1efc11c2394640. Latest consensus state: current_term: 1 leader_uuid: "127fd390c9744fb9ab1efc11c2394640" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "127fd390c9744fb9ab1efc11c2394640" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36847 } } }
I20260808 23:03:46.670907 23143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:46.671202 23148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:03:46.670841 23145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "127fd390c9744fb9ab1efc11c2394640" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "127fd390c9744fb9ab1efc11c2394640" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36847 } } }
I20260808 23:03:46.671499 23145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:46.671718 23148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:03:46.673413 23148 catalog_manager.cc:1347] Generated new cluster ID: 7f7ab0aefa5c45c0932b19e5c07dbe99
I20260808 23:03:46.673506 23148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:03:46.675061 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:46.664784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45265 (local address 127.0.0.1:36847)
0808 23:03:46.665012 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:46.665020 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:46.665048 (+ 28us) server_negotiation.cc:407] Connection header received
0808 23:03:46.665526 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:46.665534 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:46.665584 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:46.665705 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:46.666464 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:46.667380 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:46.668394 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:46.668766 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:46.668837 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:46.671373 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:46.671377 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:46.671378 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:46.671391 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:46.673432 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:46.673866 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:46.673869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:46.673870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:46.673967 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:46.674199 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:46.674200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:46.674200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:46.674467 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:46.674609 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:46.674754 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 23:03:46.674847 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":97,"threads_started":1}
I20260808 23:03:46.675966 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:03:46.684188 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:46.677448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44779 (local address 127.0.0.1:36847)
0808 23:03:46.677481 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:46.677483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:46.677577 (+ 94us) server_negotiation.cc:407] Connection header received
0808 23:03:46.677616 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:46.677617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:46.677626 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:46.677696 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:46.678233 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:46.679043 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:46.679972 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:46.680307 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:46.680379 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:46.681594 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:46.681596 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:46.681596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:46.681603 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:46.683095 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:46.683463 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:46.683464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:46.683464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:46.683520 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:46.683710 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:46.683711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:46.683711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:46.683831 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:46.683946 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:46.684041 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 23:03:46.684101 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 23:03:46.718216 23148 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:03:46.718765 23148 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:03:46.723433 23148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 127fd390c9744fb9ab1efc11c2394640: Generated new TSK 0
I20260808 23:03:46.723608 23148 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:03:46.729624 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 23035
W20260808 23:03:46.742365 23017 connection.cc:535] client connection to 127.0.0.1:36847 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36847): Connection reset by peer (error 104)
2026-08-08T23:03:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1436 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:03:46 dist-test-slave-f0tn krb5kdc[23183](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:03:46 dist-test-slave-f0tn krb5kdc[23183](info): set up 2 sockets
Aug 08 23:03:46 dist-test-slave-f0tn krb5kdc[23183](info): commencing operation
krb5kdc: starting...
W20260808 23:03:47.857674 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:03:47 dist-test-slave-f0tn krb5kdc[23183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:03:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:03:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:03:47.967797 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40019
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:48.047817 23201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:48.048010 23201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:48.048032 23201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:48.049139 23201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:03:48.049187 23201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:48.049201 23201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:48.049224 23201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:03:48.049247 23201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:03:48.050719 23201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40019
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:48.050944 23201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:48.051141 23201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:48.055969 23209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:48.055976 23207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:48.055974 23211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:48.056109 23201 server_base.cc:1034] running on GCE node
I20260808 23:03:48.056444 23201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:48.056735 23201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:48.057868 23201 hybrid_clock.cc:648] HybridClock initialized: now 1786230228057849 us; error 32 us; skew 500 ppm
Aug 08 23:03:48 dist-test-slave-f0tn krb5kdc[23183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230228, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:48.060196 23201 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:03:48.060617 23201 webserver.cc:460] Webserver started at http://127.0.0.1:39403/ using document root <none> and password file <none>
I20260808 23:03:48.060823 23201 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:48.060866 23201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:48.060987 23201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:48.062074 23201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "47bf8950b2cc47fe9877ea26ba52d8f4"
format_stamp: "Formatted at 2026-08-08 23:03:48 on dist-test-slave-f0tn"
server_key: "4082752483bc07b8f497fc69011b7b75"
server_key_iv: "f718e8a36b5b8314b7b3b28b5ce72c3e"
server_key_version: "encryptionkey@0"
I20260808 23:03:48.062393 23201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "47bf8950b2cc47fe9877ea26ba52d8f4"
format_stamp: "Formatted at 2026-08-08 23:03:48 on dist-test-slave-f0tn"
server_key: "4082752483bc07b8f497fc69011b7b75"
server_key_iv: "f718e8a36b5b8314b7b3b28b5ce72c3e"
server_key_version: "encryptionkey@0"
I20260808 23:03:48.063598 23201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:03:48.064232 23220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:48.064391 23201 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:48.064477 23201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "47bf8950b2cc47fe9877ea26ba52d8f4"
format_stamp: "Formatted at 2026-08-08 23:03:48 on dist-test-slave-f0tn"
server_key: "4082752483bc07b8f497fc69011b7b75"
server_key_iv: "f718e8a36b5b8314b7b3b28b5ce72c3e"
server_key_version: "encryptionkey@0"
I20260808 23:03:48.064531 23201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:48.080071 23201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:48.081918 23201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:48.082029 23201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:48.085726 23201 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40019
I20260808 23:03:48.085772 23307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40019 every 8 connection(s)
I20260808 23:03:48.086084 23201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:03:48.086933 23308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:48.089267 23308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4: Bootstrap starting.
I20260808 23:03:48.089871 23308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:48.090129 23308 log.cc:824] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:48.090801 23308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4: No bootstrap required, opened a new log
I20260808 23:03:48.092492 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 23201
I20260808 23:03:48.092449 23308 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf8950b2cc47fe9877ea26ba52d8f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40019 } }
I20260808 23:03:48.092619 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:03:48.092626 23308 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:48.092669 23308 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47bf8950b2cc47fe9877ea26ba52d8f4, State: Initialized, Role: FOLLOWER
I20260808 23:03:48.092808 24328 external_mini_cluster.cc:1428] Setting key 6aa85f0ea9962d92debdd6432b31515f
I20260808 23:03:48.092814 23308 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf8950b2cc47fe9877ea26ba52d8f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40019 } }
I20260808 23:03:48.092898 23308 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:03:48.092937 23308 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:03:48.092986 23308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:48.093645 23308 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf8950b2cc47fe9877ea26ba52d8f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40019 } }
I20260808 23:03:48.093752 23308 leader_election.cc:302] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47bf8950b2cc47fe9877ea26ba52d8f4; no voters:
I20260808 23:03:48.093914 23308 leader_election.cc:288] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:03:48.093948 23315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:48.094079 23315 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4 [term 1 LEADER]: Becoming Leader. State: Replica: 47bf8950b2cc47fe9877ea26ba52d8f4, State: Running, Role: LEADER
I20260808 23:03:48.094163 23308 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:03:48.094229 23315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf8950b2cc47fe9877ea26ba52d8f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40019 } }
I20260808 23:03:48.094569 23315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47bf8950b2cc47fe9877ea26ba52d8f4. Latest consensus state: current_term: 1 leader_uuid: "47bf8950b2cc47fe9877ea26ba52d8f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf8950b2cc47fe9877ea26ba52d8f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40019 } } }
I20260808 23:03:48.094655 23315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:48.094805 23320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47bf8950b2cc47fe9877ea26ba52d8f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf8950b2cc47fe9877ea26ba52d8f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40019 } } }
I20260808 23:03:48.094875 23320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4 [sys.catalog]: This master's current role is: LEADER
W20260808 23:03:48.095896 23342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:03:48.095937 23342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:03:48.095976 23329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:03:48.096125 23329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:03:48.097728 23329 catalog_manager.cc:1347] Generated new cluster ID: 8c9c502d0e254acbac88a67548fd770a
I20260808 23:03:48.097780 23329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:03:48.102249 23314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:48.093299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58943 (local address 127.0.0.1:40019)
0808 23:03:48.093804 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:48.093814 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:48.093848 (+ 34us) server_negotiation.cc:407] Connection header received
0808 23:03:48.093967 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:48.093974 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:48.094034 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:48.094436 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:48.094526 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:48.095961 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:48.097002 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:48.097562 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:48.097679 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:48.101571 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:48.101574 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:48.101575 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:48.101772 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:48.101810 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:48.101824 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:48.101901 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:48.102028 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 23:03:48.102053 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":399,"threads_started":1}
I20260808 23:03:48.103407 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:03:48.104422 23329 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:03:48.105048 23329 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:03:48.108341 23314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:48.104887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54531 (local address 127.0.0.1:40019)
0808 23:03:48.104919 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:48.104920 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:48.105014 (+ 94us) server_negotiation.cc:407] Connection header received
0808 23:03:48.105060 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:48.105061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:48.105071 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:48.105197 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:48.105343 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:48.106096 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:48.107126 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:48.107432 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:48.107481 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:48.108112 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:48.108113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:48.108113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:48.108117 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:48.108130 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:48.108140 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:48.108188 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:48.108257 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 23:03:48.108268 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 23:03:48.111317 23329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47bf8950b2cc47fe9877ea26ba52d8f4: Generated new TSK 0
I20260808 23:03:48.111466 23329 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:03:48.114019 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 23201
W20260808 23:03:48.129948 23180 connection.cc:535] client connection to 127.0.0.1:40019 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40019): Connection reset by peer (error 104)
2026-08-08T23:03:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1387 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:03:48 dist-test-slave-f0tn krb5kdc[23355](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:03:48 dist-test-slave-f0tn krb5kdc[23355](info): set up 2 sockets
Aug 08 23:03:48 dist-test-slave-f0tn krb5kdc[23355](info): commencing operation
krb5kdc: starting...
W20260808 23:03:49.256769 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:03:49 dist-test-slave-f0tn krb5kdc[23355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:03:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:03:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:03:49.370265 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40115
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:49.452230 23377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:49.452445 23377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:49.452481 23377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:49.453658 23377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:03:49.453701 23377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:49.453724 23377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:49.453744 23377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:03:49.453763 23377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:03:49.455365 23377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40115
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:49.455610 23377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:49.455832 23377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:49.460559 23390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:49.460590 23385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:49.460669 23377 server_base.cc:1034] running on GCE node
W20260808 23:03:49.460585 23386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:49.460964 23377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:49.461229 23377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:49.462376 23377 hybrid_clock.cc:648] HybridClock initialized: now 1786230229462354 us; error 35 us; skew 500 ppm
Aug 08 23:03:49 dist-test-slave-f0tn krb5kdc[23355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230229, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:49.464708 23377 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:03:49.465113 23377 webserver.cc:460] Webserver started at http://127.0.0.1:36737/ using document root <none> and password file <none>
I20260808 23:03:49.465302 23377 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:49.465340 23377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:49.465461 23377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:49.466390 23377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "c03b7ecaf3e6439a8a9045f654f33b98"
format_stamp: "Formatted at 2026-08-08 23:03:49 on dist-test-slave-f0tn"
server_key: "d9d6a1f2d29ebb12855ccc86ea08861b"
server_key_iv: "897403e56a8b51d95fe7bd712691916f"
server_key_version: "encryptionkey@0"
I20260808 23:03:49.466665 23377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "c03b7ecaf3e6439a8a9045f654f33b98"
format_stamp: "Formatted at 2026-08-08 23:03:49 on dist-test-slave-f0tn"
server_key: "d9d6a1f2d29ebb12855ccc86ea08861b"
server_key_iv: "897403e56a8b51d95fe7bd712691916f"
server_key_version: "encryptionkey@0"
I20260808 23:03:49.467864 23377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:03:49.468519 23403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:49.468693 23377 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:49.468766 23377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "c03b7ecaf3e6439a8a9045f654f33b98"
format_stamp: "Formatted at 2026-08-08 23:03:49 on dist-test-slave-f0tn"
server_key: "d9d6a1f2d29ebb12855ccc86ea08861b"
server_key_iv: "897403e56a8b51d95fe7bd712691916f"
server_key_version: "encryptionkey@0"
I20260808 23:03:49.468833 23377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:49.471982 23377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:49.473664 23377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:49.473781 23377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:49.477800 23377 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40115
I20260808 23:03:49.477846 23494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40115 every 8 connection(s)
I20260808 23:03:49.478134 23377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:03:49.479113 23497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:49.481411 23497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98: Bootstrap starting.
I20260808 23:03:49.482008 23497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:49.482256 23497 log.cc:824] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:49.482940 23497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98: No bootstrap required, opened a new log
I20260808 23:03:49.484733 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 23377
I20260808 23:03:49.484864 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:03:49.484823 23497 raft_consensus.cc:348] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03b7ecaf3e6439a8a9045f654f33b98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } }
I20260808 23:03:49.485067 24328 external_mini_cluster.cc:1428] Setting key f3fc8bd8f8b49138af76e6acc022ac31
I20260808 23:03:49.485059 23497 raft_consensus.cc:374] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:49.485100 23497 raft_consensus.cc:729] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c03b7ecaf3e6439a8a9045f654f33b98, State: Initialized, Role: FOLLOWER
I20260808 23:03:49.485245 23497 consensus_queue.cc:260] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03b7ecaf3e6439a8a9045f654f33b98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } }
I20260808 23:03:49.485316 23497 raft_consensus.cc:388] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:03:49.485340 23497 raft_consensus.cc:482] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:03:49.485409 23497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:49.486488 23497 raft_consensus.cc:504] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03b7ecaf3e6439a8a9045f654f33b98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } }
I20260808 23:03:49.486600 23497 leader_election.cc:302] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c03b7ecaf3e6439a8a9045f654f33b98; no voters:
I20260808 23:03:49.486765 23497 leader_election.cc:288] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:03:49.486836 23504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:49.487015 23504 raft_consensus.cc:686] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98 [term 1 LEADER]: Becoming Leader. State: Replica: c03b7ecaf3e6439a8a9045f654f33b98, State: Running, Role: LEADER
I20260808 23:03:49.487052 23497 sys_catalog.cc:564] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:03:49.487210 23504 consensus_queue.cc:237] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03b7ecaf3e6439a8a9045f654f33b98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } }
I20260808 23:03:49.487545 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:49.485625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38571 (local address 127.0.0.1:40115)
0808 23:03:49.485964 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:49.485975 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:49.486008 (+ 33us) server_negotiation.cc:407] Connection header received
0808 23:03:49.486150 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:49.486157 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:49.486189 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:49.486331 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:49.486996 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:49.486999 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:49.487000 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:49.487004 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:49.487051 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:49.487066 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:49.487273 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:49.487309 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 23:03:49.487348 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":87,"threads_started":1}
I20260808 23:03:49.487605 23505 sys_catalog.cc:455] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c03b7ecaf3e6439a8a9045f654f33b98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03b7ecaf3e6439a8a9045f654f33b98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } } }
I20260808 23:03:49.487677 23505 sys_catalog.cc:458] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:49.487926 23508 sys_catalog.cc:455] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c03b7ecaf3e6439a8a9045f654f33b98. Latest consensus state: current_term: 1 leader_uuid: "c03b7ecaf3e6439a8a9045f654f33b98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03b7ecaf3e6439a8a9045f654f33b98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } } }
I20260808 23:03:49.487987 23508 sys_catalog.cc:458] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:49.488473 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 23:03:49.488998 23527 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:03:49.489054 23527 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:03:49.489126 23516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:03:49.489295 23516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:03:49.490963 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:49.489894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40301 (local address 127.0.0.1:40115)
0808 23:03:49.489926 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:49.489928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:49.490115 (+ 187us) server_negotiation.cc:407] Connection header received
0808 23:03:49.490160 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:49.490161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:49.490178 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:49.490235 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:49.490762 (+ 527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:49.490763 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:49.490764 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:49.490768 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:49.490782 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:49.490786 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:49.490844 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:49.490894 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 23:03:49.490906 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 23:03:49.491003 23516 catalog_manager.cc:1347] Generated new cluster ID: 780d57a1c5164f369bec47a5eb8239e2
I20260808 23:03:49.491048 23516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:03:49.503787 23516 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:03:49.503945 23516 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:03:49.509472 23516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c03b7ecaf3e6439a8a9045f654f33b98: Generated new TSK 0
I20260808 23:03:49.509649 23516 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:03:49.515419 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 23377
2026-08-08T23:03:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1406 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:03:49 dist-test-slave-f0tn krb5kdc[23543](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:03:49 dist-test-slave-f0tn krb5kdc[23543](info): set up 2 sockets
Aug 08 23:03:49 dist-test-slave-f0tn krb5kdc[23543](info): commencing operation
krb5kdc: starting...
W20260808 23:03:50.678742 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:03:50 dist-test-slave-f0tn krb5kdc[23543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:03:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:03:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:03:50.789171 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40681
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:50.867756 23560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:50.867945 23560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:50.867966 23560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:50.869029 23560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:03:50.869062 23560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:50.869076 23560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:50.869087 23560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:03:50.869100 23560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:03:50.870505 23560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:50.870694 23560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:50.870860 23560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:50.875419 23569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:50.875428 23573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:50.875463 23560 server_base.cc:1034] running on GCE node
W20260808 23:03:50.875428 23568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:50.875813 23560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:50.876085 23560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:50.877219 23560 hybrid_clock.cc:648] HybridClock initialized: now 1786230230877203 us; error 33 us; skew 500 ppm
Aug 08 23:03:50 dist-test-slave-f0tn krb5kdc[23543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230230, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:50.879491 23560 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:03:50.879959 23560 webserver.cc:460] Webserver started at http://127.0.0.1:43739/ using document root <none> and password file <none>
I20260808 23:03:50.880149 23560 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:50.880193 23560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:50.880309 23560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:50.881256 23560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "d165245a4a1449a9a6720fe8f52b53f5"
format_stamp: "Formatted at 2026-08-08 23:03:50 on dist-test-slave-f0tn"
server_key: "5e463ca98c4878202a9c2a372b296454"
server_key_iv: "b1baea19cc6e76bef7786e71a180f9a5"
server_key_version: "encryptionkey@0"
I20260808 23:03:50.881596 23560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "d165245a4a1449a9a6720fe8f52b53f5"
format_stamp: "Formatted at 2026-08-08 23:03:50 on dist-test-slave-f0tn"
server_key: "5e463ca98c4878202a9c2a372b296454"
server_key_iv: "b1baea19cc6e76bef7786e71a180f9a5"
server_key_version: "encryptionkey@0"
I20260808 23:03:50.882784 23560 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:03:50.883404 23590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:50.883543 23560 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:50.883625 23560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "d165245a4a1449a9a6720fe8f52b53f5"
format_stamp: "Formatted at 2026-08-08 23:03:50 on dist-test-slave-f0tn"
server_key: "5e463ca98c4878202a9c2a372b296454"
server_key_iv: "b1baea19cc6e76bef7786e71a180f9a5"
server_key_version: "encryptionkey@0"
I20260808 23:03:50.883692 23560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:50.901542 23560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:50.903199 23560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:50.903328 23560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:50.907073 23560 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40681
I20260808 23:03:50.907104 23663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40681 every 8 connection(s)
I20260808 23:03:50.907414 23560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:03:50.908207 23664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:50.910346 23664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5: Bootstrap starting.
I20260808 23:03:50.910892 23664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:50.911152 23664 log.cc:824] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:50.911769 23664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5: No bootstrap required, opened a new log
I20260808 23:03:50.913334 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 23560
I20260808 23:03:50.913327 23664 raft_consensus.cc:348] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d165245a4a1449a9a6720fe8f52b53f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40681 } }
I20260808 23:03:50.913466 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:03:50.913497 23664 raft_consensus.cc:374] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:50.913544 23664 raft_consensus.cc:729] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d165245a4a1449a9a6720fe8f52b53f5, State: Initialized, Role: FOLLOWER
I20260808 23:03:50.913642 24328 external_mini_cluster.cc:1428] Setting key 746c1683a662520a00b6001d01034e7e
I20260808 23:03:50.913687 23664 consensus_queue.cc:260] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d165245a4a1449a9a6720fe8f52b53f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40681 } }
I20260808 23:03:50.913767 23664 raft_consensus.cc:388] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:03:50.913805 23664 raft_consensus.cc:482] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:03:50.913853 23664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:50.914783 23664 raft_consensus.cc:504] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d165245a4a1449a9a6720fe8f52b53f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40681 } }
I20260808 23:03:50.914897 23664 leader_election.cc:302] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d165245a4a1449a9a6720fe8f52b53f5; no voters:
I20260808 23:03:50.915057 23664 leader_election.cc:288] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:03:50.915082 23670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:50.915220 23670 raft_consensus.cc:686] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5 [term 1 LEADER]: Becoming Leader. State: Replica: d165245a4a1449a9a6720fe8f52b53f5, State: Running, Role: LEADER
I20260808 23:03:50.915328 23664 sys_catalog.cc:564] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:03:50.915309 23670 consensus_queue.cc:237] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d165245a4a1449a9a6720fe8f52b53f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40681 } }
I20260808 23:03:50.915643 23670 sys_catalog.cc:455] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d165245a4a1449a9a6720fe8f52b53f5. Latest consensus state: current_term: 1 leader_uuid: "d165245a4a1449a9a6720fe8f52b53f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d165245a4a1449a9a6720fe8f52b53f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40681 } } }
I20260808 23:03:50.915711 23670 sys_catalog.cc:458] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:50.915874 23671 sys_catalog.cc:455] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d165245a4a1449a9a6720fe8f52b53f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d165245a4a1449a9a6720fe8f52b53f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40681 } } }
I20260808 23:03:50.915937 23671 sys_catalog.cc:458] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:50.916642 23682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:03:50.917173 23682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:03:50.919100 23682 catalog_manager.cc:1347] Generated new cluster ID: 61781439ff744883912abe7e7b54949d
I20260808 23:03:50.919153 23682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 23:03:50 dist-test-slave-f0tn krb5kdc[23543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 23:03:50.923877 23668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:50.914077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58335 (local address 127.0.0.1:40681)
0808 23:03:50.914251 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:50.914261 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:50.914293 (+ 32us) server_negotiation.cc:407] Connection header received
0808 23:03:50.914386 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:50.914392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:50.914430 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:50.914532 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:50.915323 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:50.917113 (+ 1790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:50.917962 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:50.918560 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:50.918675 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:50.920982 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:50.920986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:50.920987 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:50.920997 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:50.922505 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:50.922826 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:50.922828 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:50.922829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:50.922921 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:50.923096 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:50.923097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:50.923098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:50.923316 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:50.923468 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:50.923564 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 23:03:50.923709 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
I20260808 23:03:50.924520 23682 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:03:50.924816 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:03:50.925172 23682 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:03:50.929435 23682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d165245a4a1449a9a6720fe8f52b53f5: Generated new TSK 0
I20260808 23:03:50.929590 23682 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:03:50.931617 23668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:50.926164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57099 (local address 127.0.0.1:40681)
0808 23:03:50.926202 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:50.926204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:50.926269 (+ 65us) server_negotiation.cc:407] Connection header received
0808 23:03:50.926320 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:50.926321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:50.926331 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:50.926375 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:50.926845 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:50.927317 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:50.928006 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:50.928224 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:50.928275 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:50.929363 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:50.929365 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:50.929365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:50.929371 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:50.930660 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:50.930951 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:50.930952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:50.930952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:50.931002 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:50.931167 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:50.931168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:50.931168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:50.931263 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:50.931344 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:50.931467 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 23:03:50.931537 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 23:03:50.933292 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 23560
2026-08-08T23:03:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1412 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:03:50 dist-test-slave-f0tn krb5kdc[23718](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:03:50 dist-test-slave-f0tn krb5kdc[23718](info): set up 2 sockets
Aug 08 23:03:50 dist-test-slave-f0tn krb5kdc[23718](info): commencing operation
krb5kdc: starting...
W20260808 23:03:52.069654 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:03:52 dist-test-slave-f0tn krb5kdc[23718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:03:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:03:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:03:52.179800 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40257
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:52.258213 23742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:52.258404 23742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:52.258433 23742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:52.259548 23742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:03:52.259589 23742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:52.259606 23742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:52.259622 23742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:03:52.259636 23742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:03:52.261155 23742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:52.261409 23742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:52.261605 23742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:52.266317 23755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:52.266352 23753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:52.266402 23742 server_base.cc:1034] running on GCE node
W20260808 23:03:52.266371 23752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:52.266716 23742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:52.266973 23742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:52.268110 23742 hybrid_clock.cc:648] HybridClock initialized: now 1786230232268097 us; error 27 us; skew 500 ppm
Aug 08 23:03:52 dist-test-slave-f0tn krb5kdc[23718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230232, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:52.274878 23742 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:03:52.275314 23742 webserver.cc:460] Webserver started at http://127.0.0.1:36931/ using document root <none> and password file <none>
I20260808 23:03:52.275506 23742 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:52.275552 23742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:52.275678 23742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:52.276679 23742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "a5d6cbf3f7814f3888449a90132936de"
format_stamp: "Formatted at 2026-08-08 23:03:52 on dist-test-slave-f0tn"
server_key: "a699058da6c5a60b9a7df0290319f53c"
server_key_iv: "28936f252b2599c4ab87a63857513ef8"
server_key_version: "encryptionkey@0"
I20260808 23:03:52.276996 23742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "a5d6cbf3f7814f3888449a90132936de"
format_stamp: "Formatted at 2026-08-08 23:03:52 on dist-test-slave-f0tn"
server_key: "a699058da6c5a60b9a7df0290319f53c"
server_key_iv: "28936f252b2599c4ab87a63857513ef8"
server_key_version: "encryptionkey@0"
I20260808 23:03:52.278246 23742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 23:03:52.278873 23766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:52.279021 23742 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:03:52.279104 23742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "a5d6cbf3f7814f3888449a90132936de"
format_stamp: "Formatted at 2026-08-08 23:03:52 on dist-test-slave-f0tn"
server_key: "a699058da6c5a60b9a7df0290319f53c"
server_key_iv: "28936f252b2599c4ab87a63857513ef8"
server_key_version: "encryptionkey@0"
I20260808 23:03:52.279175 23742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:52.292263 23742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:52.293848 23742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:52.293987 23742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:52.297744 23742 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40257
I20260808 23:03:52.297791 23838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40257 every 8 connection(s)
I20260808 23:03:52.298086 23742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:03:52.298887 23839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:52.301079 23839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de: Bootstrap starting.
I20260808 23:03:52.301661 23839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:52.301932 23839 log.cc:824] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:52.302551 23839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de: No bootstrap required, opened a new log
I20260808 23:03:52.304137 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 23742
I20260808 23:03:52.304101 23839 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5d6cbf3f7814f3888449a90132936de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40257 } }
I20260808 23:03:52.304229 23839 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:52.304253 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:03:52.304275 23839 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5d6cbf3f7814f3888449a90132936de, State: Initialized, Role: FOLLOWER
I20260808 23:03:52.304401 23839 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5d6cbf3f7814f3888449a90132936de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40257 } }
I20260808 23:03:52.304427 24328 external_mini_cluster.cc:1428] Setting key 8cb32fa78cef8c21b057da032933df16
I20260808 23:03:52.304458 23839 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:03:52.304482 23839 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:03:52.304526 23839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:52.305553 23839 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5d6cbf3f7814f3888449a90132936de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40257 } }
I20260808 23:03:52.305661 23839 leader_election.cc:302] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5d6cbf3f7814f3888449a90132936de; no voters:
I20260808 23:03:52.305810 23839 leader_election.cc:288] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:03:52.305845 23846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:52.305949 23846 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de [term 1 LEADER]: Becoming Leader. State: Replica: a5d6cbf3f7814f3888449a90132936de, State: Running, Role: LEADER
I20260808 23:03:52.306062 23839 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:03:52.306082 23846 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5d6cbf3f7814f3888449a90132936de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40257 } }
I20260808 23:03:52.306449 23847 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5d6cbf3f7814f3888449a90132936de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5d6cbf3f7814f3888449a90132936de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40257 } } }
I20260808 23:03:52.306533 23847 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:52.306746 23849 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5d6cbf3f7814f3888449a90132936de. Latest consensus state: current_term: 1 leader_uuid: "a5d6cbf3f7814f3888449a90132936de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5d6cbf3f7814f3888449a90132936de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40257 } } }
I20260808 23:03:52.306862 23849 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:52.307013 23857 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:03:52.307519 23857 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:03:52.308938 23857 catalog_manager.cc:1347] Generated new cluster ID: 3daa95fb2b3846ffa5d93378e0099455
I20260808 23:03:52.308995 23857 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:03:52.310249 23844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:52.304861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41439 (local address 127.0.0.1:40257)
0808 23:03:52.305151 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:52.305160 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:52.305196 (+ 36us) server_negotiation.cc:407] Connection header received
0808 23:03:52.305311 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:52.305318 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:52.305407 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:52.305553 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:52.305753 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:52.307051 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:52.308145 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:52.308725 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:52.308830 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:52.309586 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:52.309590 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:52.309590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:52.309814 (+ 224us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:52.309852 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:52.309867 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:52.309939 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:52.309993 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 23:03:52.310054 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":180,"threads_started":1}
I20260808 23:03:52.311107 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:03:52.316184 23844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:52.312573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48433 (local address 127.0.0.1:40257)
0808 23:03:52.312604 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:52.312605 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:52.312846 (+ 241us) server_negotiation.cc:407] Connection header received
0808 23:03:52.312932 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:52.312933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:52.312943 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:52.312971 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:52.313149 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:52.313864 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:52.314897 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:52.315193 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:52.315235 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:52.315946 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:52.315947 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:52.315947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:52.315952 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:52.315964 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:52.315968 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:52.315994 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:52.316092 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 23:03:52.316118 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 23:03:52.331179 23857 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:03:52.331736 23857 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:03:52.334205 23857 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5d6cbf3f7814f3888449a90132936de: Generated new TSK 0
I20260808 23:03:52.334360 23857 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:03:52.337289 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 23742
2026-08-08T23:03:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1401 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:03:52 dist-test-slave-f0tn krb5kdc[23896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:03:52 dist-test-slave-f0tn krb5kdc[23896](info): set up 2 sockets
Aug 08 23:03:52 dist-test-slave-f0tn krb5kdc[23896](info): commencing operation
krb5kdc: starting...
W20260808 23:03:53.469431 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:03:53 dist-test-slave-f0tn krb5kdc[23896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:03:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:03:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:03:53.581854 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45637
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:53.662014 23913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:53.662230 23913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:53.662264 23913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:53.663475 23913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:03:53.663525 23913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:53.663549 23913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:53.663570 23913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:03:53.663599 23913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:03:53.665153 23913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:53.665447 23913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:53.665674 23913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:53.670365 23925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:53.670397 23929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:53.670375 23923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:53.670463 23913 server_base.cc:1034] running on GCE node
I20260808 23:03:53.670781 23913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:53.671044 23913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:53.672187 23913 hybrid_clock.cc:648] HybridClock initialized: now 1786230233672170 us; error 32 us; skew 500 ppm
Aug 08 23:03:53 dist-test-slave-f0tn krb5kdc[23896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230233, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:53.674580 23913 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:03:53.674975 23913 webserver.cc:460] Webserver started at http://127.0.0.1:43663/ using document root <none> and password file <none>
I20260808 23:03:53.675201 23913 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:53.675246 23913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:53.675371 23913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:53.676398 23913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "bf25c3e85a4745418a53e4bda563b234"
format_stamp: "Formatted at 2026-08-08 23:03:53 on dist-test-slave-f0tn"
server_key: "fd9eefe5d9fb0a0ff90150495d737cc8"
server_key_iv: "c6ca343805f46c3ae2615762a42492fa"
server_key_version: "encryptionkey@0"
I20260808 23:03:53.676712 23913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "bf25c3e85a4745418a53e4bda563b234"
format_stamp: "Formatted at 2026-08-08 23:03:53 on dist-test-slave-f0tn"
server_key: "fd9eefe5d9fb0a0ff90150495d737cc8"
server_key_iv: "c6ca343805f46c3ae2615762a42492fa"
server_key_version: "encryptionkey@0"
I20260808 23:03:53.677955 23913 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:03:53.678588 23940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:53.678733 23913 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:53.678819 23913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "bf25c3e85a4745418a53e4bda563b234"
format_stamp: "Formatted at 2026-08-08 23:03:53 on dist-test-slave-f0tn"
server_key: "fd9eefe5d9fb0a0ff90150495d737cc8"
server_key_iv: "c6ca343805f46c3ae2615762a42492fa"
server_key_version: "encryptionkey@0"
I20260808 23:03:53.678879 23913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:53.682009 23913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:53.683614 23913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:53.683728 23913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:53.689285 23913 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45637
I20260808 23:03:53.689329 24028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45637 every 8 connection(s)
I20260808 23:03:53.689620 23913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:03:53.690538 24029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:53.692682 24029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234: Bootstrap starting.
I20260808 23:03:53.693223 24029 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:53.693524 24029 log.cc:824] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:53.694141 24029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234: No bootstrap required, opened a new log
I20260808 23:03:53.695688 24029 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf25c3e85a4745418a53e4bda563b234" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45637 } }
I20260808 23:03:53.695814 24029 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:53.695847 24029 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf25c3e85a4745418a53e4bda563b234, State: Initialized, Role: FOLLOWER
I20260808 23:03:53.695986 24029 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf25c3e85a4745418a53e4bda563b234" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45637 } }
I20260808 23:03:53.696061 24029 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:03:53.696095 24029 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:03:53.696141 24029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:53.696642 24029 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf25c3e85a4745418a53e4bda563b234" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45637 } }
I20260808 23:03:53.696729 24029 leader_election.cc:302] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf25c3e85a4745418a53e4bda563b234; no voters:
I20260808 23:03:53.696875 24029 leader_election.cc:288] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:03:53.696907 24034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:53.697034 24034 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234 [term 1 LEADER]: Becoming Leader. State: Replica: bf25c3e85a4745418a53e4bda563b234, State: Running, Role: LEADER
I20260808 23:03:53.697113 24029 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:03:53.697154 24034 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf25c3e85a4745418a53e4bda563b234" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45637 } }
I20260808 23:03:53.697582 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 23913
I20260808 23:03:53.697625 24038 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf25c3e85a4745418a53e4bda563b234" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf25c3e85a4745418a53e4bda563b234" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45637 } } }
I20260808 23:03:53.697710 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:03:53.697708 24038 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:53.697918 24328 external_mini_cluster.cc:1428] Setting key d7b4c5cff3d12025d32b7a63775956e2
I20260808 23:03:53.698179 24039 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf25c3e85a4745418a53e4bda563b234. Latest consensus state: current_term: 1 leader_uuid: "bf25c3e85a4745418a53e4bda563b234" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf25c3e85a4745418a53e4bda563b234" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45637 } } }
I20260808 23:03:53.698292 24039 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:53.698413 24055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:03:53.698930 24055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:03:53.700300 24073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:53.698541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48105 (local address 127.0.0.1:45637)
0808 23:03:53.698722 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:53.698731 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:53.698765 (+ 34us) server_negotiation.cc:407] Connection header received
0808 23:03:53.698871 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:53.698876 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:53.698898 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:53.699011 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:53.699726 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:53.699730 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:53.699730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:53.699735 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:53.699780 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:53.699795 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:53.699871 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:53.700065 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 23:03:53.700107 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260808 23:03:53.700402 24055 catalog_manager.cc:1347] Generated new cluster ID: d8dbd8a029e24acfbfb4f9dc78570cb9
I20260808 23:03:53.700462 24055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:03:53.701152 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:03:53.703498 24073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:53.702570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44433 (local address 127.0.0.1:45637)
0808 23:03:53.702602 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:53.702604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:53.702632 (+ 28us) server_negotiation.cc:407] Connection header received
0808 23:03:53.702717 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:53.702718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:53.702722 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:53.702769 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:53.703304 (+ 535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:53.703305 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:53.703305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:53.703309 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:53.703323 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:53.703327 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:53.703367 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:53.703430 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 23:03:53.703441 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 23:03:53.732769 24055 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:03:53.732959 24055 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:03:53.743005 24055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf25c3e85a4745418a53e4bda563b234: Generated new TSK 0
I20260808 23:03:53.743235 24055 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:03:53.753021 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 23913
2026-08-08T23:03:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1421 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:03:53 dist-test-slave-f0tn krb5kdc[24085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:03:53 dist-test-slave-f0tn krb5kdc[24085](info): set up 2 sockets
Aug 08 23:03:53 dist-test-slave-f0tn krb5kdc[24085](info): commencing operation
krb5kdc: starting...
W20260808 23:03:54.889400 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:03:54 dist-test-slave-f0tn krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:03:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:03:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:03:54.999881 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34533
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:55.079032 24106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:55.079246 24106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:55.079269 24106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:55.080355 24106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:03:55.080387 24106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:55.080400 24106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:55.080411 24106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:03:55.080422 24106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:03:55.081903 24106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:55.082108 24106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:55.082286 24106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:55.086896 24116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:55.086987 24115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:55.086956 24106 server_base.cc:1034] running on GCE node
W20260808 23:03:55.086905 24119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:55.087361 24106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:55.087634 24106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:55.088784 24106 hybrid_clock.cc:648] HybridClock initialized: now 1786230235088764 us; error 31 us; skew 500 ppm
Aug 08 23:03:55 dist-test-slave-f0tn krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230235, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:55.094758 24106 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:03:55.095176 24106 webserver.cc:460] Webserver started at http://127.0.0.1:46707/ using document root <none> and password file <none>
I20260808 23:03:55.095366 24106 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:55.095405 24106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:55.095502 24106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:55.096470 24106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "14d8457d4e66473bada86ec261c90a18"
format_stamp: "Formatted at 2026-08-08 23:03:55 on dist-test-slave-f0tn"
server_key: "d70b9eba9ae4ff7682eccb711dcda440"
server_key_iv: "b1f31a2010249ed5991e64faa79e5808"
server_key_version: "encryptionkey@0"
I20260808 23:03:55.096796 24106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "14d8457d4e66473bada86ec261c90a18"
format_stamp: "Formatted at 2026-08-08 23:03:55 on dist-test-slave-f0tn"
server_key: "d70b9eba9ae4ff7682eccb711dcda440"
server_key_iv: "b1f31a2010249ed5991e64faa79e5808"
server_key_version: "encryptionkey@0"
I20260808 23:03:55.097990 24106 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 23:03:55.098644 24126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:55.098795 24106 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:55.098881 24106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "14d8457d4e66473bada86ec261c90a18"
format_stamp: "Formatted at 2026-08-08 23:03:55 on dist-test-slave-f0tn"
server_key: "d70b9eba9ae4ff7682eccb711dcda440"
server_key_iv: "b1f31a2010249ed5991e64faa79e5808"
server_key_version: "encryptionkey@0"
I20260808 23:03:55.098948 24106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:55.112383 24106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:55.113996 24106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:55.114114 24106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:55.118018 24106 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34533
I20260808 23:03:55.118048 24209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34533 every 8 connection(s)
I20260808 23:03:55.118352 24106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:03:55.119168 24213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:55.121330 24213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18: Bootstrap starting.
I20260808 23:03:55.121913 24213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:55.122180 24213 log.cc:824] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:55.122793 24213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18: No bootstrap required, opened a new log
I20260808 23:03:55.123986 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 24106
I20260808 23:03:55.124090 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:03:55.124258 24328 external_mini_cluster.cc:1428] Setting key fd21b490b0ced55ca8c6e15b37e78e6a
I20260808 23:03:55.124444 24213 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d8457d4e66473bada86ec261c90a18" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34533 } }
I20260808 23:03:55.124572 24213 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:55.124604 24213 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14d8457d4e66473bada86ec261c90a18, State: Initialized, Role: FOLLOWER
I20260808 23:03:55.124764 24213 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d8457d4e66473bada86ec261c90a18" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34533 } }
I20260808 23:03:55.124907 24213 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:03:55.124963 24213 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:03:55.125008 24213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:55.126078 24213 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d8457d4e66473bada86ec261c90a18" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34533 } }
I20260808 23:03:55.126189 24213 leader_election.cc:302] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14d8457d4e66473bada86ec261c90a18; no voters:
I20260808 23:03:55.126348 24213 leader_election.cc:288] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:03:55.126377 24221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:55.126521 24221 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18 [term 1 LEADER]: Becoming Leader. State: Replica: 14d8457d4e66473bada86ec261c90a18, State: Running, Role: LEADER
I20260808 23:03:55.126614 24213 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:03:55.126663 24221 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d8457d4e66473bada86ec261c90a18" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34533 } }
I20260808 23:03:55.126988 24222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14d8457d4e66473bada86ec261c90a18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d8457d4e66473bada86ec261c90a18" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34533 } } }
I20260808 23:03:55.127074 24222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:55.127228 24223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14d8457d4e66473bada86ec261c90a18. Latest consensus state: current_term: 1 leader_uuid: "14d8457d4e66473bada86ec261c90a18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d8457d4e66473bada86ec261c90a18" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34533 } } }
I20260808 23:03:55.127279 24223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:55.127691 24233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:03:55.128175 24233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:03:55.129715 24233 catalog_manager.cc:1347] Generated new cluster ID: 8050c27cba07423a9e3c967226fb91ba
I20260808 23:03:55.129765 24233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 23:03:55 dist-test-slave-f0tn krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 23:03:55.136699 24218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:55.124799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:51121 (local address 127.0.0.1:34533)
0808 23:03:55.124956 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:55.124966 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:55.125031 (+ 65us) server_negotiation.cc:407] Connection header received
0808 23:03:55.125147 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:55.125152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:55.125198 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:55.125295 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:55.126095 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:55.127227 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:55.129276 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:55.129845 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:55.129950 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:55.132620 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:55.132624 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:55.132624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:55.132637 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:55.135117 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:55.135505 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:55.135508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:55.135508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:55.135625 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:55.135833 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:55.135834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:55.135834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:55.136128 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:55.136300 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:55.136379 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 23:03:55.136481 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260808 23:03:55.137677 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:03:55.145916 24218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:55.139131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:60529 (local address 127.0.0.1:34533)
0808 23:03:55.139164 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:55.139165 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:55.139368 (+ 203us) server_negotiation.cc:407] Connection header received
0808 23:03:55.139425 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:55.139426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:55.139436 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:55.139483 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:55.140052 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:55.140772 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:55.141760 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:55.142063 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:55.142116 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:55.143382 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:55.143384 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:55.143384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:55.143390 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:55.144810 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:55.145189 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:55.145189 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:55.145190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:55.145235 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:55.145452 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:55.145453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:55.145453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:55.145560 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:55.145655 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:55.145779 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 23:03:55.145836 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 23:03:55.159140 24233 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:03:55.159827 24233 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:03:55.166328 24233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14d8457d4e66473bada86ec261c90a18: Generated new TSK 0
I20260808 23:03:55.166525 24233 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:03:55.173262 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 24106
W20260808 23:03:55.186254 24083 connection.cc:535] client connection to 127.0.0.1:34533 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34533): Connection reset by peer (error 104)
2026-08-08T23:03:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1415 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:03:55 dist-test-slave-f0tn krb5kdc[24268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:03:55 dist-test-slave-f0tn krb5kdc[24268](info): set up 2 sockets
Aug 08 23:03:55 dist-test-slave-f0tn krb5kdc[24268](info): commencing operation
krb5kdc: starting...
W20260808 23:03:56.315022 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:03:56 dist-test-slave-f0tn krb5kdc[24268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:03:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:03:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:03:56.425165 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36163
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:56.503690 24299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:56.503876 24299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:56.503897 24299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:56.504979 24299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:03:56.505019 24299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:56.505033 24299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:56.505055 24299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:03:56.505079 24299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:03:56.506522 24299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:56.506734 24299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:56.506934 24299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:56.511545 24306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:56.511632 24299 server_base.cc:1034] running on GCE node
W20260808 23:03:56.511591 24313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:56.511718 24309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:56.512012 24299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:56.512300 24299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:56.513413 24299 hybrid_clock.cc:648] HybridClock initialized: now 1786230236513400 us; error 30 us; skew 500 ppm
Aug 08 23:03:56 dist-test-slave-f0tn krb5kdc[24268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230236, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:56.515882 24299 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:03:56.516335 24299 webserver.cc:460] Webserver started at http://127.0.0.1:38875/ using document root <none> and password file <none>
I20260808 23:03:56.516654 24299 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:56.516767 24299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:56.516914 24299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:56.518253 24299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "cff590544f6d4648bf9255df157d8bf7"
format_stamp: "Formatted at 2026-08-08 23:03:56 on dist-test-slave-f0tn"
server_key: "0e0f7f6d559c59909872bfb05fa4d44c"
server_key_iv: "02372f380548596d3a89881610a22760"
server_key_version: "encryptionkey@0"
I20260808 23:03:56.518694 24299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "cff590544f6d4648bf9255df157d8bf7"
format_stamp: "Formatted at 2026-08-08 23:03:56 on dist-test-slave-f0tn"
server_key: "0e0f7f6d559c59909872bfb05fa4d44c"
server_key_iv: "02372f380548596d3a89881610a22760"
server_key_version: "encryptionkey@0"
I20260808 23:03:56.520205 24299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:03:56.525825 24320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:56.525995 24299 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260808 23:03:56.526080 24299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "cff590544f6d4648bf9255df157d8bf7"
format_stamp: "Formatted at 2026-08-08 23:03:56 on dist-test-slave-f0tn"
server_key: "0e0f7f6d559c59909872bfb05fa4d44c"
server_key_iv: "02372f380548596d3a89881610a22760"
server_key_version: "encryptionkey@0"
I20260808 23:03:56.526149 24299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:56.550648 24299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:56.552754 24299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:56.552858 24299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:56.556586 24299 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36163
I20260808 23:03:56.556631 24411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36163 every 8 connection(s)
I20260808 23:03:56.556958 24299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:03:56.557812 24412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:56.559319 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 24299
I20260808 23:03:56.559445 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:03:56.559669 24328 external_mini_cluster.cc:1428] Setting key 242555477fb673bab258959a758efe66
I20260808 23:03:56.560375 24412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7: Bootstrap starting.
I20260808 23:03:56.560940 24412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:56.561205 24412 log.cc:824] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:56.561897 24412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7: No bootstrap required, opened a new log
I20260808 23:03:56.563618 24412 raft_consensus.cc:348] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cff590544f6d4648bf9255df157d8bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36163 } }
I20260808 23:03:56.563751 24412 raft_consensus.cc:374] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:56.563783 24412 raft_consensus.cc:729] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cff590544f6d4648bf9255df157d8bf7, State: Initialized, Role: FOLLOWER
I20260808 23:03:56.563911 24412 consensus_queue.cc:260] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cff590544f6d4648bf9255df157d8bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36163 } }
I20260808 23:03:56.563977 24412 raft_consensus.cc:388] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:03:56.564000 24412 raft_consensus.cc:482] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:03:56.564042 24412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:56.564584 24412 raft_consensus.cc:504] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cff590544f6d4648bf9255df157d8bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36163 } }
I20260808 23:03:56.564695 24412 leader_election.cc:302] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cff590544f6d4648bf9255df157d8bf7; no voters:
I20260808 23:03:56.564771 24415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:56.560168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:49061 (local address 127.0.0.1:36163)
0808 23:03:56.560347 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:56.560355 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:56.560379 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:03:56.560842 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:56.560848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:56.560896 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:56.561018 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:56.561269 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:56.562336 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:56.563087 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:56.563475 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:56.563568 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:56.564204 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:56.564208 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:56.564209 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:56.564357 (+ 148us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:56.564386 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:56.564396 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:56.564462 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:56.564558 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 23:03:56.564585 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260808 23:03:56.564855 24412 leader_election.cc:288] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:03:56.565011 24418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:56.565088 24412 sys_catalog.cc:564] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:03:56.565186 24418 raft_consensus.cc:686] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7 [term 1 LEADER]: Becoming Leader. State: Replica: cff590544f6d4648bf9255df157d8bf7, State: Running, Role: LEADER
I20260808 23:03:56.565265 24418 consensus_queue.cc:237] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cff590544f6d4648bf9255df157d8bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36163 } }
I20260808 23:03:56.565626 24417 sys_catalog.cc:455] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cff590544f6d4648bf9255df157d8bf7. Latest consensus state: current_term: 1 leader_uuid: "cff590544f6d4648bf9255df157d8bf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cff590544f6d4648bf9255df157d8bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36163 } } }
I20260808 23:03:56.565685 24419 sys_catalog.cc:455] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cff590544f6d4648bf9255df157d8bf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cff590544f6d4648bf9255df157d8bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36163 } } }
I20260808 23:03:56.565783 24417 sys_catalog.cc:458] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:56.565865 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:03:56.565798 24419 sys_catalog.cc:458] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7 [sys.catalog]: This master's current role is: LEADER
W20260808 23:03:56.566895 24441 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:03:56.567000 24441 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:03:56.567102 24432 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:03:56.567302 24432 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:03:56.568768 24432 catalog_manager.cc:1347] Generated new cluster ID: f94ecc314707496183d15aac33fc6ba2
I20260808 23:03:56.568823 24432 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:03:56.570315 24415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:56.567378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:49701 (local address 127.0.0.1:36163)
0808 23:03:56.567411 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:56.567413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:56.567590 (+ 177us) server_negotiation.cc:407] Connection header received
0808 23:03:56.567629 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:56.567630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:56.567646 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:56.567689 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:56.567805 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:56.568430 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:56.569210 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:56.569482 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:56.569543 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:56.570096 (+ 553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:56.570098 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:56.570098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:56.570102 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:56.570116 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:56.570120 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:56.570166 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:56.570233 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 23:03:56.570242 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 23:03:56.594398 24432 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:03:56.594928 24432 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:03:56.605816 24432 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cff590544f6d4648bf9255df157d8bf7: Generated new TSK 0
I20260808 23:03:56.606020 24432 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:03:56.613323 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 24299
W20260808 23:03:56.626583 24265 connection.cc:535] client connection to 127.0.0.1:36163 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36163): Connection reset by peer (error 104)
2026-08-08T23:03:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1439 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:03:56 dist-test-slave-f0tn krb5kdc[24461](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:03:56 dist-test-slave-f0tn krb5kdc[24461](info): set up 2 sockets
Aug 08 23:03:56 dist-test-slave-f0tn krb5kdc[24461](info): commencing operation
krb5kdc: starting...
W20260808 23:03:57.755187 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:03:57 dist-test-slave-f0tn krb5kdc[24461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:03:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:03:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:03:57.868011 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40527
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:57.948050 24480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:57.948240 24480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:57.948261 24480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:57.949441 24480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:03:57.949482 24480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:57.949498 24480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:57.949518 24480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:03:57.949540 24480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:03:57.951040 24480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:57.951262 24480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:57.951455 24480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:57.956071 24489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:57.956084 24487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:57.956168 24480 server_base.cc:1034] running on GCE node
W20260808 23:03:57.956071 24486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:57.956585 24480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:57.956846 24480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:57.957980 24480 hybrid_clock.cc:648] HybridClock initialized: now 1786230237957969 us; error 28 us; skew 500 ppm
Aug 08 23:03:57 dist-test-slave-f0tn krb5kdc[24461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230237, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:57.963826 24480 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:03:57.964262 24480 webserver.cc:460] Webserver started at http://127.0.0.1:41921/ using document root <none> and password file <none>
I20260808 23:03:57.964464 24480 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:57.964504 24480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:57.964597 24480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:57.965651 24480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "a090ace24d1949af8434ee0601b063d5"
format_stamp: "Formatted at 2026-08-08 23:03:57 on dist-test-slave-f0tn"
server_key: "08130b8fb39038344e6dc7ec2a554198"
server_key_iv: "cf952b5046b56782a0d7203cb0343c19"
server_key_version: "encryptionkey@0"
I20260808 23:03:57.965958 24480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "a090ace24d1949af8434ee0601b063d5"
format_stamp: "Formatted at 2026-08-08 23:03:57 on dist-test-slave-f0tn"
server_key: "08130b8fb39038344e6dc7ec2a554198"
server_key_iv: "cf952b5046b56782a0d7203cb0343c19"
server_key_version: "encryptionkey@0"
I20260808 23:03:57.967137 24480 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:03:57.967808 24500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:57.967964 24480 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:03:57.968053 24480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "a090ace24d1949af8434ee0601b063d5"
format_stamp: "Formatted at 2026-08-08 23:03:57 on dist-test-slave-f0tn"
server_key: "08130b8fb39038344e6dc7ec2a554198"
server_key_iv: "cf952b5046b56782a0d7203cb0343c19"
server_key_version: "encryptionkey@0"
I20260808 23:03:57.968113 24480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:57.971330 24480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:57.973030 24480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:57.973152 24480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:57.977485 24480 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40527
I20260808 23:03:57.977522 24592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40527 every 8 connection(s)
I20260808 23:03:57.977825 24480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:03:57.978621 24593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:57.980772 24593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5: Bootstrap starting.
I20260808 23:03:57.981298 24593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:57.981587 24593 log.cc:824] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:57.982230 24593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5: No bootstrap required, opened a new log
I20260808 23:03:57.982235 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 24480
I20260808 23:03:57.982347 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:03:57.982504 24328 external_mini_cluster.cc:1428] Setting key 223921a599ba121e6447edc6007f6bb2
I20260808 23:03:57.984177 24593 raft_consensus.cc:348] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a090ace24d1949af8434ee0601b063d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40527 } }
I20260808 23:03:57.984341 24593 raft_consensus.cc:374] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:57.984378 24593 raft_consensus.cc:729] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a090ace24d1949af8434ee0601b063d5, State: Initialized, Role: FOLLOWER
I20260808 23:03:57.984547 24593 consensus_queue.cc:260] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a090ace24d1949af8434ee0601b063d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40527 } }
I20260808 23:03:57.984648 24593 raft_consensus.cc:388] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:03:57.984705 24593 raft_consensus.cc:482] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:03:57.984756 24593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:57.985365 24601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:57.982951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:34717 (local address 127.0.0.1:40527)
0808 23:03:57.983168 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:57.983177 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:57.983210 (+ 33us) server_negotiation.cc:407] Connection header received
0808 23:03:57.984158 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:57.984167 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:57.984201 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:57.984348 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:57.984922 (+ 574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:57.984926 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:57.984926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:57.984931 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:57.984974 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:57.984987 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:57.985061 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:57.985096 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 23:03:57.985145 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":84,"threads_started":1}
I20260808 23:03:57.985765 24593 raft_consensus.cc:504] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a090ace24d1949af8434ee0601b063d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40527 } }
I20260808 23:03:57.985854 24593 leader_election.cc:302] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a090ace24d1949af8434ee0601b063d5; no voters:
I20260808 23:03:57.986004 24593 leader_election.cc:288] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:03:57.986037 24602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:57.986320 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:03:57.986282 24593 sys_catalog.cc:564] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:03:57.986269 24602 raft_consensus.cc:686] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5 [term 1 LEADER]: Becoming Leader. State: Replica: a090ace24d1949af8434ee0601b063d5, State: Running, Role: LEADER
I20260808 23:03:57.987589 24602 consensus_queue.cc:237] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a090ace24d1949af8434ee0601b063d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40527 } }
I20260808 23:03:57.988229 24603 sys_catalog.cc:455] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a090ace24d1949af8434ee0601b063d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a090ace24d1949af8434ee0601b063d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40527 } } }
I20260808 23:03:57.988303 24603 sys_catalog.cc:458] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:57.988101 24604 sys_catalog.cc:455] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a090ace24d1949af8434ee0601b063d5. Latest consensus state: current_term: 1 leader_uuid: "a090ace24d1949af8434ee0601b063d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a090ace24d1949af8434ee0601b063d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40527 } } }
I20260808 23:03:57.988631 24604 sys_catalog.cc:458] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5 [sys.catalog]: This master's current role is: LEADER
W20260808 23:03:57.988655 24631 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:03:57.988690 24631 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:03:57.988960 24633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:03:57.989120 24633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:03:57.989774 24601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:57.988063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:35243 (local address 127.0.0.1:40527)
0808 23:03:57.988095 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:57.988097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:57.988846 (+ 749us) server_negotiation.cc:407] Connection header received
0808 23:03:57.988886 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:57.988886 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:57.988891 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:57.988944 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:57.989557 (+ 613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:57.989559 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:57.989559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:03:57.989563 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:57.989576 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:03:57.989580 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:57.989607 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:57.989704 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 23:03:57.989717 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 23:03:57.990599 24633 catalog_manager.cc:1347] Generated new cluster ID: a19a9b035fe3495a8ff08ef64b1aa8eb
I20260808 23:03:57.990649 24633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:03:57.997035 24633 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:03:57.997182 24633 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:03:58.005172 24633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a090ace24d1949af8434ee0601b063d5: Generated new TSK 0
I20260808 23:03:58.005340 24633 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:03:58.007972 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 24480
2026-08-08T23:03:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1394 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:03:58 dist-test-slave-f0tn krb5kdc[24645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:03:58 dist-test-slave-f0tn krb5kdc[24645](info): set up 2 sockets
Aug 08 23:03:58 dist-test-slave-f0tn krb5kdc[24645](info): commencing operation
krb5kdc: starting...
W20260808 23:03:59.143642 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:03:59 dist-test-slave-f0tn krb5kdc[24645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:03:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:03:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:03:59.255985 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36535
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:03:59.335008 24663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:03:59.335212 24663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:03:59.335234 24663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:03:59.336293 24663 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:03:59.336324 24663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:03:59.336337 24663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:03:59.336349 24663 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:03:59.336362 24663 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:03:59.337812 24663 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36535
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:03:59.338006 24663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:03:59.338184 24663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:03:59.342757 24680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:59.342784 24676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:03:59.342784 24678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:03:59.342881 24663 server_base.cc:1034] running on GCE node
I20260808 23:03:59.343140 24663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:03:59.343408 24663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:03:59.344547 24663 hybrid_clock.cc:648] HybridClock initialized: now 1786230239344521 us; error 44 us; skew 500 ppm
Aug 08 23:03:59 dist-test-slave-f0tn krb5kdc[24645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230239, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:03:59.346879 24663 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:03:59.347270 24663 webserver.cc:460] Webserver started at http://127.0.0.1:39433/ using document root <none> and password file <none>
I20260808 23:03:59.347477 24663 fs_manager.cc:359] Metadata directory not provided
I20260808 23:03:59.347522 24663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:03:59.347648 24663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:03:59.348626 24663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "a97b07f97f704b1eb43f48b586bf4e85"
format_stamp: "Formatted at 2026-08-08 23:03:59 on dist-test-slave-f0tn"
server_key: "2ff3a19e15212a103c21dc64769ec6c7"
server_key_iv: "6df0ccff16150ddee23e1ac05f51c108"
server_key_version: "encryptionkey@0"
I20260808 23:03:59.348958 24663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "a97b07f97f704b1eb43f48b586bf4e85"
format_stamp: "Formatted at 2026-08-08 23:03:59 on dist-test-slave-f0tn"
server_key: "2ff3a19e15212a103c21dc64769ec6c7"
server_key_iv: "6df0ccff16150ddee23e1ac05f51c108"
server_key_version: "encryptionkey@0"
I20260808 23:03:59.350214 24663 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 23:03:59.350852 24690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:03:59.350996 24663 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:03:59.351075 24663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "a97b07f97f704b1eb43f48b586bf4e85"
format_stamp: "Formatted at 2026-08-08 23:03:59 on dist-test-slave-f0tn"
server_key: "2ff3a19e15212a103c21dc64769ec6c7"
server_key_iv: "6df0ccff16150ddee23e1ac05f51c108"
server_key_version: "encryptionkey@0"
I20260808 23:03:59.351138 24663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:03:59.364301 24663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:03:59.365923 24663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:03:59.366050 24663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:03:59.369831 24663 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36535
I20260808 23:03:59.369863 24773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36535 every 8 connection(s)
I20260808 23:03:59.370162 24663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:03:59.370957 24774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:03:59.373135 24774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85: Bootstrap starting.
I20260808 23:03:59.373724 24774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85: Neither blocks nor log segments found. Creating new log.
I20260808 23:03:59.373998 24774 log.cc:824] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85: Log is configured to *not* fsync() on all Append() calls
I20260808 23:03:59.374636 24774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85: No bootstrap required, opened a new log
I20260808 23:03:59.376209 24774 raft_consensus.cc:348] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97b07f97f704b1eb43f48b586bf4e85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36535 } }
I20260808 23:03:59.376340 24774 raft_consensus.cc:374] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:03:59.376371 24774 raft_consensus.cc:729] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a97b07f97f704b1eb43f48b586bf4e85, State: Initialized, Role: FOLLOWER
I20260808 23:03:59.376502 24774 consensus_queue.cc:260] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97b07f97f704b1eb43f48b586bf4e85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36535 } }
I20260808 23:03:59.376559 24774 raft_consensus.cc:388] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:03:59.376581 24774 raft_consensus.cc:482] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:03:59.376627 24774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:03:59.377130 24774 raft_consensus.cc:504] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97b07f97f704b1eb43f48b586bf4e85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36535 } }
I20260808 23:03:59.377228 24774 leader_election.cc:302] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a97b07f97f704b1eb43f48b586bf4e85; no voters:
I20260808 23:03:59.377413 24774 leader_election.cc:288] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:03:59.377439 24777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:03:59.377565 24777 raft_consensus.cc:686] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85 [term 1 LEADER]: Becoming Leader. State: Replica: a97b07f97f704b1eb43f48b586bf4e85, State: Running, Role: LEADER
I20260808 23:03:59.377663 24774 sys_catalog.cc:564] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:03:59.377682 24777 consensus_queue.cc:237] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97b07f97f704b1eb43f48b586bf4e85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36535 } }
I20260808 23:03:59.378023 24780 sys_catalog.cc:455] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a97b07f97f704b1eb43f48b586bf4e85. Latest consensus state: current_term: 1 leader_uuid: "a97b07f97f704b1eb43f48b586bf4e85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97b07f97f704b1eb43f48b586bf4e85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36535 } } }
I20260808 23:03:59.378134 24779 sys_catalog.cc:455] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a97b07f97f704b1eb43f48b586bf4e85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97b07f97f704b1eb43f48b586bf4e85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36535 } } }
I20260808 23:03:59.378168 24780 sys_catalog.cc:458] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:59.378247 24779 sys_catalog.cc:458] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85 [sys.catalog]: This master's current role is: LEADER
I20260808 23:03:59.378587 24788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:03:59.379069 24788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:03:59.380295 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 24663
I20260808 23:03:59.380421 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:03:59.380599 24328 external_mini_cluster.cc:1428] Setting key 05d98bb43f0b003a160bf64e5cb4eced
I20260808 23:03:59.380596 24788 catalog_manager.cc:1347] Generated new cluster ID: 26efa521c12842f88589d4c2cb835d30
I20260808 23:03:59.380649 24788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 23:03:59 dist-test-slave-f0tn krb5kdc[24645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 23:03:59.388705 24788 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:03:59.389415 24788 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:03:59.389940 24806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:59.380992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:46561 (local address 127.0.0.1:36535)
0808 23:03:59.381476 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:59.381483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:59.381508 (+ 25us) server_negotiation.cc:407] Connection header received
0808 23:03:59.381593 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:59.381598 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:59.381638 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:59.381768 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:59.382373 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:59.383277 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:59.384003 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:59.384430 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:59.384522 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:59.386728 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:59.386732 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:59.386733 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:59.386747 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:59.388231 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:59.388791 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:59.388795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:59.388795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:59.388888 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:59.389091 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:59.389092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:59.389092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:59.389348 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:59.389545 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:59.389626 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 23:03:59.389739 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":441,"thread_start_us":72,"threads_started":1}
I20260808 23:03:59.390805 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:03:59.394484 24788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a97b07f97f704b1eb43f48b586bf4e85: Generated new TSK 0
I20260808 23:03:59.394631 24788 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:03:59.399591 24806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:03:59.392061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:56577 (local address 127.0.0.1:36535)
0808 23:03:59.392114 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 23:03:59.392116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:03:59.393540 (+ 1424us) server_negotiation.cc:407] Connection header received
0808 23:03:59.393596 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:03:59.393597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:03:59.393622 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:03:59.393668 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:03:59.394239 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:59.394912 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:59.395701 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:03:59.395933 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:03:59.396003 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:03:59.397152 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:03:59.397153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:03:59.397154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:03:59.397161 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:03:59.398401 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:59.398865 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:59.398866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:59.398866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:59.398919 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:03:59.399128 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:03:59.399129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:03:59.399129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:03:59.399227 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:03:59.399314 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 23:03:59.399427 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 23:03:59.399484 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 23:03:59.401269 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 24663
W20260808 23:03:59.413424 24640 connection.cc:535] client connection to 127.0.0.1:36535 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36535): Connection reset by peer (error 104)
2026-08-08T23:03:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1392 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:03:59 dist-test-slave-f0tn krb5kdc[24818](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:03:59 dist-test-slave-f0tn krb5kdc[24818](info): set up 2 sockets
Aug 08 23:03:59 dist-test-slave-f0tn krb5kdc[24818](info): commencing operation
krb5kdc: starting...
W20260808 23:04:00.539453 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:04:00 dist-test-slave-f0tn krb5kdc[24818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:04:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:04:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:04:00.651844 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37513
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:04:00.731498 24836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:04:00.731700 24836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:04:00.731732 24836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:04:00.732842 24836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:04:00.732882 24836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:04:00.732900 24836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:04:00.732916 24836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:04:00.732932 24836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:04:00.734422 24836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37513
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:04:00.734656 24836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:04:00.734862 24836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:04:00.739472 24845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:04:00.739516 24836 server_base.cc:1034] running on GCE node
W20260808 23:04:00.739457 24842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:00.739457 24841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:04:00.739872 24836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:04:00.740154 24836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:04:00.741282 24836 hybrid_clock.cc:648] HybridClock initialized: now 1786230240741271 us; error 31 us; skew 500 ppm
Aug 08 23:04:00 dist-test-slave-f0tn krb5kdc[24818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230240, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:04:00.743623 24836 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:04:00.744024 24836 webserver.cc:460] Webserver started at http://127.0.0.1:44751/ using document root <none> and password file <none>
I20260808 23:04:00.744243 24836 fs_manager.cc:359] Metadata directory not provided
I20260808 23:04:00.744292 24836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:04:00.744414 24836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:04:00.745430 24836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "e27d2d459bf74b35876268a9d163fbeb"
format_stamp: "Formatted at 2026-08-08 23:04:00 on dist-test-slave-f0tn"
server_key: "4352c11716be05e5616be6d3e85305eb"
server_key_iv: "837aa5863931cfe2dc65965c3f1f46a7"
server_key_version: "encryptionkey@0"
I20260808 23:04:00.745759 24836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "e27d2d459bf74b35876268a9d163fbeb"
format_stamp: "Formatted at 2026-08-08 23:04:00 on dist-test-slave-f0tn"
server_key: "4352c11716be05e5616be6d3e85305eb"
server_key_iv: "837aa5863931cfe2dc65965c3f1f46a7"
server_key_version: "encryptionkey@0"
I20260808 23:04:00.746934 24836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 23:04:00.747597 24856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:04:00.747754 24836 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:04:00.747838 24836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "e27d2d459bf74b35876268a9d163fbeb"
format_stamp: "Formatted at 2026-08-08 23:04:00 on dist-test-slave-f0tn"
server_key: "4352c11716be05e5616be6d3e85305eb"
server_key_iv: "837aa5863931cfe2dc65965c3f1f46a7"
server_key_version: "encryptionkey@0"
I20260808 23:04:00.747901 24836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:04:00.776690 24836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:04:00.778900 24836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:04:00.779043 24836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:04:00.782972 24836 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37513
I20260808 23:04:00.783012 24935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37513 every 8 connection(s)
I20260808 23:04:00.783352 24836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:04:00.784215 24936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:04:00.786175 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 24836
I20260808 23:04:00.786293 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:04:00.786464 24328 external_mini_cluster.cc:1428] Setting key 6978eb3d3c942fcf4b41ccf9c2792fc1
I20260808 23:04:00.786545 24936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb: Bootstrap starting.
I20260808 23:04:00.787276 24936 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb: Neither blocks nor log segments found. Creating new log.
I20260808 23:04:00.787613 24936 log.cc:824] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb: Log is configured to *not* fsync() on all Append() calls
I20260808 23:04:00.788378 24936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb: No bootstrap required, opened a new log
I20260808 23:04:00.790619 24936 raft_consensus.cc:348] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e27d2d459bf74b35876268a9d163fbeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37513 } }
I20260808 23:04:00.790786 24936 raft_consensus.cc:374] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:04:00.790827 24936 raft_consensus.cc:729] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e27d2d459bf74b35876268a9d163fbeb, State: Initialized, Role: FOLLOWER
I20260808 23:04:00.790954 24936 consensus_queue.cc:260] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e27d2d459bf74b35876268a9d163fbeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37513 } }
I20260808 23:04:00.791039 24936 raft_consensus.cc:388] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:04:00.791069 24936 raft_consensus.cc:482] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:04:00.791170 24936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:04:00.791759 24936 raft_consensus.cc:504] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e27d2d459bf74b35876268a9d163fbeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37513 } }
I20260808 23:04:00.791872 24936 leader_election.cc:302] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e27d2d459bf74b35876268a9d163fbeb; no voters:
I20260808 23:04:00.792037 24936 leader_election.cc:288] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:04:00.792079 24943 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:04:00.792258 24943 raft_consensus.cc:686] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb [term 1 LEADER]: Becoming Leader. State: Replica: e27d2d459bf74b35876268a9d163fbeb, State: Running, Role: LEADER
I20260808 23:04:00.792313 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:00.786939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:52865 (local address 127.0.0.1:37513)
0808 23:04:00.787130 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:00.787139 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:00.787174 (+ 35us) server_negotiation.cc:407] Connection header received
0808 23:04:00.787741 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:00.787750 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:00.787820 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:00.787943 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:00.788181 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:00.789551 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:00.790310 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:00.790905 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:00.791010 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:00.791604 (+ 594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:00.791609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:00.791609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:04:00.791818 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:00.791859 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:04:00.791874 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:00.791960 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:00.792038 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 23:04:00.792066 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":87,"threads_started":1}
I20260808 23:04:00.792336 24936 sys_catalog.cc:564] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:04:00.793347 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:04:00.792392 24943 consensus_queue.cc:237] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e27d2d459bf74b35876268a9d163fbeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37513 } }
I20260808 23:04:00.793929 24945 sys_catalog.cc:455] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e27d2d459bf74b35876268a9d163fbeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e27d2d459bf74b35876268a9d163fbeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37513 } } }
I20260808 23:04:00.794013 24945 sys_catalog.cc:458] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb [sys.catalog]: This master's current role is: LEADER
W20260808 23:04:00.794261 24965 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:04:00.794314 24965 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:04:00.794364 24944 sys_catalog.cc:455] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e27d2d459bf74b35876268a9d163fbeb. Latest consensus state: current_term: 1 leader_uuid: "e27d2d459bf74b35876268a9d163fbeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e27d2d459bf74b35876268a9d163fbeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37513 } } }
I20260808 23:04:00.794428 24944 sys_catalog.cc:458] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb [sys.catalog]: This master's current role is: LEADER
I20260808 23:04:00.794909 24968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:04:00.795105 24968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:04:00.796665 24968 catalog_manager.cc:1347] Generated new cluster ID: 3197e5b406cd493cbd92af3530a68778
I20260808 23:04:00.796716 24968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:04:00.798060 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:00.794865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:53549 (local address 127.0.0.1:37513)
0808 23:04:00.794898 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:00.794900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:00.795125 (+ 225us) server_negotiation.cc:407] Connection header received
0808 23:04:00.795171 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:00.795172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:00.795181 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:00.795236 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:00.795360 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:00.796039 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:00.796883 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:00.797184 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:00.797254 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:00.797832 (+ 578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:00.797833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:00.797834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:04:00.797838 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:00.797852 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:04:00.797855 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:00.797920 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:00.797982 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 23:04:00.797991 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 23:04:00.828176 24968 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:04:00.828679 24968 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:04:00.833673 24968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e27d2d459bf74b35876268a9d163fbeb: Generated new TSK 0
I20260808 23:04:00.833834 24968 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:04:00.837957 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 24836
W20260808 23:04:00.859696 24816 connection.cc:535] client connection to 127.0.0.1:37513 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37513): Connection reset by peer (error 104)
2026-08-08T23:04:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1446 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:04:00 dist-test-slave-f0tn krb5kdc[24985](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:04:00 dist-test-slave-f0tn krb5kdc[24985](info): set up 2 sockets
Aug 08 23:04:00 dist-test-slave-f0tn krb5kdc[24985](info): commencing operation
krb5kdc: starting...
W20260808 23:04:01.978495 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:04:02 dist-test-slave-f0tn krb5kdc[24985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:04:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:04:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:04:02.091782 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44639
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:04:02.171077 25001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:04:02.171295 25001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:04:02.171327 25001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:04:02.172435 25001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:04:02.172477 25001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:04:02.172494 25001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:04:02.172510 25001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:04:02.172528 25001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:04:02.173982 25001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44639
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:04:02.174230 25001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:04:02.174435 25001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 23:04:02.179141 25001 server_base.cc:1034] running on GCE node
W20260808 23:04:02.179149 25011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:02.179122 25009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:02.179126 25014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:04:02.179494 25001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:04:02.179766 25001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:04:02.180907 25001 hybrid_clock.cc:648] HybridClock initialized: now 1786230242180888 us; error 33 us; skew 500 ppm
Aug 08 23:04:02 dist-test-slave-f0tn krb5kdc[24985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230242, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:04:02.183246 25001 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:04:02.183656 25001 webserver.cc:460] Webserver started at http://127.0.0.1:36003/ using document root <none> and password file <none>
I20260808 23:04:02.183851 25001 fs_manager.cc:359] Metadata directory not provided
I20260808 23:04:02.183898 25001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:04:02.184021 25001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:04:02.185074 25001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "230c7c96e7ff4462a290af62c98ace9f"
format_stamp: "Formatted at 2026-08-08 23:04:02 on dist-test-slave-f0tn"
server_key: "dba1f2b87e0d47ccbf9b6a8f13386773"
server_key_iv: "9ddcea20deec12ee3b702cb2076bb612"
server_key_version: "encryptionkey@0"
I20260808 23:04:02.185463 25001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "230c7c96e7ff4462a290af62c98ace9f"
format_stamp: "Formatted at 2026-08-08 23:04:02 on dist-test-slave-f0tn"
server_key: "dba1f2b87e0d47ccbf9b6a8f13386773"
server_key_iv: "9ddcea20deec12ee3b702cb2076bb612"
server_key_version: "encryptionkey@0"
I20260808 23:04:02.186653 25001 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 23:04:02.187323 25027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:04:02.187469 25001 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:04:02.187552 25001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "230c7c96e7ff4462a290af62c98ace9f"
format_stamp: "Formatted at 2026-08-08 23:04:02 on dist-test-slave-f0tn"
server_key: "dba1f2b87e0d47ccbf9b6a8f13386773"
server_key_iv: "9ddcea20deec12ee3b702cb2076bb612"
server_key_version: "encryptionkey@0"
I20260808 23:04:02.187618 25001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:04:02.190771 25001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:04:02.192376 25001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:04:02.192509 25001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:04:02.196264 25001 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44639
I20260808 23:04:02.196301 25108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44639 every 8 connection(s)
I20260808 23:04:02.196576 25001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:04:02.197504 25109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:04:02.199858 25109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f: Bootstrap starting.
I20260808 23:04:02.200393 25109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f: Neither blocks nor log segments found. Creating new log.
I20260808 23:04:02.200641 25109 log.cc:824] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f: Log is configured to *not* fsync() on all Append() calls
I20260808 23:04:02.201273 25109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f: No bootstrap required, opened a new log
I20260808 23:04:02.202829 25109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230c7c96e7ff4462a290af62c98ace9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44639 } }
I20260808 23:04:02.202952 25109 raft_consensus.cc:374] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:04:02.202984 25109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 230c7c96e7ff4462a290af62c98ace9f, State: Initialized, Role: FOLLOWER
I20260808 23:04:02.203114 25109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230c7c96e7ff4462a290af62c98ace9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44639 } }
I20260808 23:04:02.203178 25109 raft_consensus.cc:388] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:04:02.203212 25109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:04:02.203259 25109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:04:02.203755 25109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230c7c96e7ff4462a290af62c98ace9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44639 } }
I20260808 23:04:02.203843 25109 leader_election.cc:302] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 230c7c96e7ff4462a290af62c98ace9f; no voters:
I20260808 23:04:02.203984 25109 leader_election.cc:288] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:04:02.204018 25118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:04:02.204174 25118 raft_consensus.cc:686] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f [term 1 LEADER]: Becoming Leader. State: Replica: 230c7c96e7ff4462a290af62c98ace9f, State: Running, Role: LEADER
I20260808 23:04:02.204208 25109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:04:02.204263 25118 consensus_queue.cc:237] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230c7c96e7ff4462a290af62c98ace9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44639 } }
I20260808 23:04:02.204605 25122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 230c7c96e7ff4462a290af62c98ace9f. Latest consensus state: current_term: 1 leader_uuid: "230c7c96e7ff4462a290af62c98ace9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230c7c96e7ff4462a290af62c98ace9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44639 } } }
I20260808 23:04:02.204698 25122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f [sys.catalog]: This master's current role is: LEADER
I20260808 23:04:02.204707 25119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "230c7c96e7ff4462a290af62c98ace9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230c7c96e7ff4462a290af62c98ace9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44639 } } }
I20260808 23:04:02.204772 25119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f [sys.catalog]: This master's current role is: LEADER
I20260808 23:04:02.204957 25131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:04:02.205500 25131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:04:02.205878 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 25001
I20260808 23:04:02.206002 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:04:02.206176 24328 external_mini_cluster.cc:1428] Setting key f18bd89254276de695b140a539124d59
I20260808 23:04:02.207243 25131 catalog_manager.cc:1347] Generated new cluster ID: 69f2045946594473a5eef532557dd0df
I20260808 23:04:02.207293 25131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:04:02.208349 25145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:02.206751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:53901 (local address 127.0.0.1:44639)
0808 23:04:02.206994 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:02.207003 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:02.207029 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:04:02.207114 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:02.207119 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:02.207153 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:02.207287 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:02.207952 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:02.207956 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:02.207956 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:04:02.207961 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:02.208010 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:04:02.208021 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:02.208071 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:02.208138 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 23:04:02.208180 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":74,"threads_started":1}
I20260808 23:04:02.209117 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:04:02.211267 25145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:02.210403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.119:33149 (local address 127.0.0.1:44639)
0808 23:04:02.210466 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:02.210467 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:02.210503 (+ 36us) server_negotiation.cc:407] Connection header received
0808 23:04:02.210563 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:02.210564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:02.210569 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:02.210609 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:02.211081 (+ 472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:02.211082 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:02.211082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:04:02.211086 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:02.211099 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:04:02.211102 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:02.211148 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:02.211198 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 23:04:02.211209 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 23:04:02.219727 25131 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:04:02.219882 25131 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:04:02.239486 25131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 230c7c96e7ff4462a290af62c98ace9f: Generated new TSK 0
I20260808 23:04:02.239691 25131 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:04:02.246284 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 25001
2026-08-08T23:04:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1396 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:04:02 dist-test-slave-f0tn krb5kdc[25160](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:04:02 dist-test-slave-f0tn krb5kdc[25160](info): set up 2 sockets
Aug 08 23:04:02 dist-test-slave-f0tn krb5kdc[25160](info): commencing operation
krb5kdc: starting...
W20260808 23:04:03.383765 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:04:03 dist-test-slave-f0tn krb5kdc[25160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:04:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:04:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:04:03.493342 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:35515
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:38631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:35515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:04:03.571825 25177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:04:03.572011 25177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:04:03.572043 25177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:04:03.573136 25177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:04:03.573181 25177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:04:03.573201 25177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:04:03.573217 25177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:04:03.573233 25177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:04:03.574713 25177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:38631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:35515
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:35515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:04:03.574940 25177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:04:03.575129 25177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:04:03.579591 25187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:04:03.579687 25177 server_base.cc:1034] running on GCE node
W20260808 23:04:03.579586 25183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:03.579620 25185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:04:03.580044 25177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:04:03.580312 25177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:04:03.581427 25177 hybrid_clock.cc:648] HybridClock initialized: now 1786230243581411 us; error 32 us; skew 500 ppm
Aug 08 23:04:03 dist-test-slave-f0tn krb5kdc[25160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230243, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:04:03.597884 25177 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:04:03.598316 25177 webserver.cc:460] Webserver started at http://127.23.194.62:38145/ using document root <none> and password file <none>
I20260808 23:04:03.598510 25177 fs_manager.cc:359] Metadata directory not provided
I20260808 23:04:03.598556 25177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:04:03.598682 25177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:04:03.599691 25177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "98406533aa14492a82adc769594ebef9"
format_stamp: "Formatted at 2026-08-08 23:04:03 on dist-test-slave-f0tn"
server_key: "94e4fabfd1cb437d6bd5e7b1bd138b92"
server_key_iv: "b08f7c6e4d7bce590c241d1065435bef"
server_key_version: "encryptionkey@0"
I20260808 23:04:03.600026 25177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "98406533aa14492a82adc769594ebef9"
format_stamp: "Formatted at 2026-08-08 23:04:03 on dist-test-slave-f0tn"
server_key: "94e4fabfd1cb437d6bd5e7b1bd138b92"
server_key_iv: "b08f7c6e4d7bce590c241d1065435bef"
server_key_version: "encryptionkey@0"
I20260808 23:04:03.601269 25177 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 23:04:03.601969 25198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:04:03.602114 25177 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:04:03.602228 25177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "98406533aa14492a82adc769594ebef9"
format_stamp: "Formatted at 2026-08-08 23:04:03 on dist-test-slave-f0tn"
server_key: "94e4fabfd1cb437d6bd5e7b1bd138b92"
server_key_iv: "b08f7c6e4d7bce590c241d1065435bef"
server_key_version: "encryptionkey@0"
I20260808 23:04:03.602290 25177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:04:03.620398 25177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:04:03.622555 25177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:04:03.622694 25177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:04:03.626507 25177 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:35515
I20260808 23:04:03.626550 25274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:35515 every 8 connection(s)
I20260808 23:04:03.626873 25177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:04:03.627511 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 25177
I20260808 23:04:03.627627 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:04:03.627729 25275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:04:03.627801 24328 external_mini_cluster.cc:1428] Setting key beced095fbe1695741ffcd9b9739a1b8
I20260808 23:04:03.631026 25275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9: Bootstrap starting.
I20260808 23:04:03.631628 25275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9: Neither blocks nor log segments found. Creating new log.
I20260808 23:04:03.631899 25275 log.cc:824] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9: Log is configured to *not* fsync() on all Append() calls
I20260808 23:04:03.632571 25275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9: No bootstrap required, opened a new log
Aug 08 23:04:03 dist-test-slave-f0tn krb5kdc[25160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:04:03.634524 25275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98406533aa14492a82adc769594ebef9" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35515 } }
I20260808 23:04:03.634690 25275 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:04:03.634727 25275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98406533aa14492a82adc769594ebef9, State: Initialized, Role: FOLLOWER
I20260808 23:04:03.634858 25275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98406533aa14492a82adc769594ebef9" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35515 } }
I20260808 23:04:03.634939 25275 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:04:03.634971 25275 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:04:03.635017 25275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:04:03.635645 25275 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98406533aa14492a82adc769594ebef9" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35515 } }
I20260808 23:04:03.635756 25275 leader_election.cc:302] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98406533aa14492a82adc769594ebef9; no voters:
I20260808 23:04:03.635908 25275 leader_election.cc:288] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:04:03.635955 25284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:04:03.636092 25284 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9 [term 1 LEADER]: Becoming Leader. State: Replica: 98406533aa14492a82adc769594ebef9, State: Running, Role: LEADER
I20260808 23:04:03.636159 25275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:04:03.636217 25284 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98406533aa14492a82adc769594ebef9" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35515 } }
I20260808 23:04:03.636670 25285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98406533aa14492a82adc769594ebef9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98406533aa14492a82adc769594ebef9" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35515 } } }
I20260808 23:04:03.636765 25285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9 [sys.catalog]: This master's current role is: LEADER
I20260808 23:04:03.636999 25287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98406533aa14492a82adc769594ebef9. Latest consensus state: current_term: 1 leader_uuid: "98406533aa14492a82adc769594ebef9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98406533aa14492a82adc769594ebef9" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 35515 } } }
I20260808 23:04:03.637127 25287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9 [sys.catalog]: This master's current role is: LEADER
I20260808 23:04:03.637300 25301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:04:03.637880 25301 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:04:03.638477 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:03.628269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33413 (local address 127.23.194.62:35515)
0808 23:04:03.628434 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:03.628444 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:03.628484 (+ 40us) server_negotiation.cc:407] Connection header received
0808 23:04:03.628884 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:03.628894 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:03.628969 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:03.629116 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:03.629722 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:03.631028 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:03.631775 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:03.632192 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:03.632277 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:03.634413 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:03.634418 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:03.634418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:03.634430 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:03.636267 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:03.636772 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:03.636775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:03.636775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:03.636883 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:03.637172 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:03.637173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:03.637173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:03.637502 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:03.637658 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:03.638182 (+ 524us) server_negotiation.cc:299] Negotiation successful
0808 23:04:03.638281 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260808 23:04:03.639384 25301 catalog_manager.cc:1347] Generated new cluster ID: b642aab8d1cb4781afeea654292a5a92
I20260808 23:04:03.639436 25301 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:04:03.641431 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:04:03.648012 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:03.642698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55287 (local address 127.23.194.62:35515)
0808 23:04:03.642729 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:03.642730 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:03.642825 (+ 95us) server_negotiation.cc:407] Connection header received
0808 23:04:03.642863 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:03.642864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:03.642873 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:03.642916 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:03.643368 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:03.643826 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:03.644469 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:03.644682 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:03.644734 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:03.645818 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:03.645819 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:03.645819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:03.645826 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:03.647009 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:03.647329 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:03.647330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:03.647330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:03.647377 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:03.647561 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:03.647561 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:03.647562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:03.647658 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:03.647742 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:03.647875 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 23:04:03.647932 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 23:04:03.657199 25301 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:04:03.657753 25301 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:04:03.660081 25301 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98406533aa14492a82adc769594ebef9: Generated new TSK 0
I20260808 23:04:03.660234 25301 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:04:03.665462 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 25177
W20260808 23:04:03.687690 25156 connection.cc:535] client connection to 127.23.194.62:35515 recv error: Network error: failed to read from TLS socket (remote: 127.23.194.62:35515): Connection reset by peer (error 104)
2026-08-08T23:04:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1431 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:04:03 dist-test-slave-f0tn krb5kdc[25333](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:04:03 dist-test-slave-f0tn krb5kdc[25333](info): set up 2 sockets
Aug 08 23:04:03 dist-test-slave-f0tn krb5kdc[25333](info): commencing operation
krb5kdc: starting...
W20260808 23:04:04.807701 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:04:04 dist-test-slave-f0tn krb5kdc[25333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:04:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:04:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:04:04.921734 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:32883
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:40497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:32883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:04:05.002755 25351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:04:05.002974 25351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:04:05.002997 25351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:04:05.004089 25351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:04:05.004127 25351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:04:05.004143 25351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:04:05.004160 25351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:04:05.004176 25351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:04:05.005684 25351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:40497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:32883
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:32883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:04:05.005893 25351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:04:05.006070 25351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:04:05.010828 25360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:04:05.010964 25351 server_base.cc:1034] running on GCE node
W20260808 23:04:05.010828 25358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:05.010813 25356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:04:05.011286 25351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:04:05.011535 25351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:04:05.012677 25351 hybrid_clock.cc:648] HybridClock initialized: now 1786230245012660 us; error 33 us; skew 500 ppm
Aug 08 23:04:05 dist-test-slave-f0tn krb5kdc[25333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230245, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:04:05.015146 25351 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:04:05.015553 25351 webserver.cc:460] Webserver started at http://127.23.194.62:45199/ using document root <none> and password file <none>
I20260808 23:04:05.015767 25351 fs_manager.cc:359] Metadata directory not provided
I20260808 23:04:05.015810 25351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:04:05.015920 25351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:04:05.016917 25351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "d210f8817c1448e490629542b24f9c6a"
format_stamp: "Formatted at 2026-08-08 23:04:05 on dist-test-slave-f0tn"
server_key: "8cb19ec9e4ef11763434e59ad12e05a0"
server_key_iv: "921cd0fc8691aca7b5c01e1662ae372a"
server_key_version: "encryptionkey@0"
I20260808 23:04:05.017231 25351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "d210f8817c1448e490629542b24f9c6a"
format_stamp: "Formatted at 2026-08-08 23:04:05 on dist-test-slave-f0tn"
server_key: "8cb19ec9e4ef11763434e59ad12e05a0"
server_key_iv: "921cd0fc8691aca7b5c01e1662ae372a"
server_key_version: "encryptionkey@0"
I20260808 23:04:05.018472 25351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:04:05.019090 25374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:04:05.019218 25351 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:04:05.019299 25351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "d210f8817c1448e490629542b24f9c6a"
format_stamp: "Formatted at 2026-08-08 23:04:05 on dist-test-slave-f0tn"
server_key: "8cb19ec9e4ef11763434e59ad12e05a0"
server_key_iv: "921cd0fc8691aca7b5c01e1662ae372a"
server_key_version: "encryptionkey@0"
I20260808 23:04:05.019363 25351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:04:05.037461 25351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:04:05.039106 25351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:04:05.039235 25351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:04:05.043337 25351 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:32883
I20260808 23:04:05.043367 25450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:32883 every 8 connection(s)
I20260808 23:04:05.043689 25351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:04:05.044487 25451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:04:05.045950 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 25351
I20260808 23:04:05.046087 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:04:05.046275 24328 external_mini_cluster.cc:1428] Setting key a69bb4e3cec53b5c1e1ecfb0fb042f8a
I20260808 23:04:05.047046 25451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a: Bootstrap starting.
I20260808 23:04:05.047618 25451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a: Neither blocks nor log segments found. Creating new log.
I20260808 23:04:05.047933 25451 log.cc:824] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a: Log is configured to *not* fsync() on all Append() calls
I20260808 23:04:05.048554 25451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a: No bootstrap required, opened a new log
I20260808 23:04:05.050817 25451 raft_consensus.cc:348] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d210f8817c1448e490629542b24f9c6a" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 32883 } }
I20260808 23:04:05.050992 25451 raft_consensus.cc:374] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:04:05.051041 25451 raft_consensus.cc:729] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d210f8817c1448e490629542b24f9c6a, State: Initialized, Role: FOLLOWER
I20260808 23:04:05.051167 25451 consensus_queue.cc:260] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d210f8817c1448e490629542b24f9c6a" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 32883 } }
I20260808 23:04:05.051230 25451 raft_consensus.cc:388] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:04:05.051250 25451 raft_consensus.cc:482] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:04:05.051295 25451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:04:05.051703 25454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:05.046764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48585 (local address 127.23.194.62:32883)
0808 23:04:05.046918 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:05.046926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:05.046948 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:04:05.047370 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:05.047377 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:05.047422 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:05.047532 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:05.047794 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:05.048753 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:05.049839 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:05.050238 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:05.050321 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:05.051009 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:05.051013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:05.051014 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:04:05.051236 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:05.051276 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:04:05.051291 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:05.051368 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:05.051478 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 23:04:05.051504 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260808 23:04:05.051887 25451 raft_consensus.cc:504] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d210f8817c1448e490629542b24f9c6a" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 32883 } }
I20260808 23:04:05.051993 25451 leader_election.cc:302] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d210f8817c1448e490629542b24f9c6a; no voters:
I20260808 23:04:05.052139 25451 leader_election.cc:288] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:04:05.052172 25460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:04:05.052316 25460 raft_consensus.cc:686] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a [term 1 LEADER]: Becoming Leader. State: Replica: d210f8817c1448e490629542b24f9c6a, State: Running, Role: LEADER
I20260808 23:04:05.052380 25451 sys_catalog.cc:564] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:04:05.052412 25460 consensus_queue.cc:237] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d210f8817c1448e490629542b24f9c6a" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 32883 } }
I20260808 23:04:05.052629 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:04:05.053445 25463 sys_catalog.cc:455] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d210f8817c1448e490629542b24f9c6a. Latest consensus state: current_term: 1 leader_uuid: "d210f8817c1448e490629542b24f9c6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d210f8817c1448e490629542b24f9c6a" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 32883 } } }
I20260808 23:04:05.053530 25463 sys_catalog.cc:458] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a [sys.catalog]: This master's current role is: LEADER
I20260808 23:04:05.054128 25465 sys_catalog.cc:455] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d210f8817c1448e490629542b24f9c6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d210f8817c1448e490629542b24f9c6a" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 32883 } } }
W20260808 23:04:05.054139 25489 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:04:05.054317 25489 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:04:05.054412 25465 sys_catalog.cc:458] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a [sys.catalog]: This master's current role is: LEADER
I20260808 23:04:05.054435 25487 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:04:05.054697 25487 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:04:05.055979 25487 catalog_manager.cc:1347] Generated new cluster ID: 0e3f352a0f84496dab095cbbda38e6de
I20260808 23:04:05.056025 25487 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:04:05.057562 25454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:05.054126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54277 (local address 127.23.194.62:32883)
0808 23:04:05.054169 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:05.054170 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:05.054365 (+ 195us) server_negotiation.cc:407] Connection header received
0808 23:04:05.054437 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:05.054438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:05.054448 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:05.054476 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:05.054684 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:05.055377 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:05.056332 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:05.056635 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:05.056705 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:05.057319 (+ 614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:05.057320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:05.057321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:04:05.057324 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:05.057336 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:04:05.057340 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:05.057423 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:05.057484 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 23:04:05.057494 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 23:04:05.069249 25487 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:04:05.069774 25487 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:04:05.080684 25487 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d210f8817c1448e490629542b24f9c6a: Generated new TSK 0
I20260808 23:04:05.080866 25487 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:04:05.090364 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 25351
W20260808 23:04:05.108974 25331 connection.cc:535] client connection to 127.23.194.62:32883 recv error: Network error: failed to read from TLS socket (remote: 127.23.194.62:32883): Connection reset by peer (error 104)
2026-08-08T23:04:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1421 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:04:05 dist-test-slave-f0tn krb5kdc[25506](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:04:05 dist-test-slave-f0tn krb5kdc[25506](info): set up 2 sockets
Aug 08 23:04:05 dist-test-slave-f0tn krb5kdc[25506](info): commencing operation
krb5kdc: starting...
W20260808 23:04:06.249856 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:04:06 dist-test-slave-f0tn krb5kdc[25506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:04:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:04:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:04:06.362025 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:37877
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:37877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:04:06.440675 25526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:04:06.440872 25526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:04:06.440903 25526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:04:06.442073 25526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:04:06.442114 25526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:04:06.442132 25526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:04:06.442155 25526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:04:06.442170 25526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:04:06.443620 25526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:37877
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:37877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:04:06.443857 25526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:04:06.444057 25526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:04:06.448568 25534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:06.448602 25535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:06.448580 25537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:04:06.448848 25526 server_base.cc:1034] running on GCE node
I20260808 23:04:06.449072 25526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:04:06.449350 25526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:04:06.450520 25526 hybrid_clock.cc:648] HybridClock initialized: now 1786230246450502 us; error 33 us; skew 500 ppm
Aug 08 23:04:06 dist-test-slave-f0tn krb5kdc[25506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230246, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:04:06.452885 25526 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:04:06.453274 25526 webserver.cc:460] Webserver started at http://127.23.194.62:45971/ using document root <none> and password file <none>
I20260808 23:04:06.453517 25526 fs_manager.cc:359] Metadata directory not provided
I20260808 23:04:06.453564 25526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:04:06.453692 25526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:04:06.454690 25526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "74f411bd05ed44ef855912e9999374b3"
format_stamp: "Formatted at 2026-08-08 23:04:06 on dist-test-slave-f0tn"
server_key: "680f48a9ab377fca94001b27cf4ce883"
server_key_iv: "482856321cf6347a873b4f255b8f14e6"
server_key_version: "encryptionkey@0"
I20260808 23:04:06.455003 25526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "74f411bd05ed44ef855912e9999374b3"
format_stamp: "Formatted at 2026-08-08 23:04:06 on dist-test-slave-f0tn"
server_key: "680f48a9ab377fca94001b27cf4ce883"
server_key_iv: "482856321cf6347a873b4f255b8f14e6"
server_key_version: "encryptionkey@0"
I20260808 23:04:06.456245 25526 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:04:06.456858 25556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:04:06.456996 25526 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:04:06.457072 25526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "74f411bd05ed44ef855912e9999374b3"
format_stamp: "Formatted at 2026-08-08 23:04:06 on dist-test-slave-f0tn"
server_key: "680f48a9ab377fca94001b27cf4ce883"
server_key_iv: "482856321cf6347a873b4f255b8f14e6"
server_key_version: "encryptionkey@0"
I20260808 23:04:06.457135 25526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:04:06.460209 25526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:04:06.462234 25526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:04:06.462358 25526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:04:06.466032 25526 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:37877
I20260808 23:04:06.466068 25641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:37877 every 8 connection(s)
I20260808 23:04:06.466336 25526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:04:06.467278 25642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:04:06.469343 25642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3: Bootstrap starting.
I20260808 23:04:06.469899 25642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3: Neither blocks nor log segments found. Creating new log.
I20260808 23:04:06.470158 25642 log.cc:824] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3: Log is configured to *not* fsync() on all Append() calls
I20260808 23:04:06.470758 25642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3: No bootstrap required, opened a new log
I20260808 23:04:06.472287 25642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74f411bd05ed44ef855912e9999374b3" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 37877 } }
I20260808 23:04:06.472409 25642 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:04:06.472440 25642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74f411bd05ed44ef855912e9999374b3, State: Initialized, Role: FOLLOWER
I20260808 23:04:06.472565 25642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74f411bd05ed44ef855912e9999374b3" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 37877 } }
I20260808 23:04:06.472622 25642 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:04:06.472645 25642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:04:06.472688 25642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:04:06.473495 25642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74f411bd05ed44ef855912e9999374b3" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 37877 } }
I20260808 23:04:06.473587 25642 leader_election.cc:302] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74f411bd05ed44ef855912e9999374b3; no voters:
I20260808 23:04:06.473739 25642 leader_election.cc:288] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:04:06.473764 25646 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:04:06.473882 25646 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3 [term 1 LEADER]: Becoming Leader. State: Replica: 74f411bd05ed44ef855912e9999374b3, State: Running, Role: LEADER
I20260808 23:04:06.473971 25642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:04:06.474009 25646 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74f411bd05ed44ef855912e9999374b3" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 37877 } }
I20260808 23:04:06.474380 25651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74f411bd05ed44ef855912e9999374b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74f411bd05ed44ef855912e9999374b3" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 37877 } } }
I20260808 23:04:06.474474 25651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3 [sys.catalog]: This master's current role is: LEADER
I20260808 23:04:06.474398 25652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74f411bd05ed44ef855912e9999374b3. Latest consensus state: current_term: 1 leader_uuid: "74f411bd05ed44ef855912e9999374b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74f411bd05ed44ef855912e9999374b3" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 37877 } } }
I20260808 23:04:06.474599 25652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3 [sys.catalog]: This master's current role is: LEADER
I20260808 23:04:06.474969 25660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:04:06.475390 25660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:04:06.476055 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 25526
I20260808 23:04:06.476168 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:04:06.476325 24328 external_mini_cluster.cc:1428] Setting key 42256283811d55e0be2a310de566c2a9
I20260808 23:04:06.477170 25660 catalog_manager.cc:1347] Generated new cluster ID: 5c752ec21a154f2091a979b503539d95
I20260808 23:04:06.477226 25660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:04:06.482220 25681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:06.476766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53689 (local address 127.23.194.62:37877)
0808 23:04:06.476948 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:06.476956 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:06.476987 (+ 31us) server_negotiation.cc:407] Connection header received
0808 23:04:06.477093 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:06.477099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:06.477130 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:06.477272 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:06.481811 (+ 4539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:06.481815 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:06.481815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:04:06.481820 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:06.481865 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:04:06.481879 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:06.481938 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:06.482009 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 23:04:06.482047 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":75,"threads_started":1}
I20260808 23:04:06.483080 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:04:06.485508 25681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:06.484474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34549 (local address 127.23.194.62:37877)
0808 23:04:06.484515 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:06.484516 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:06.484626 (+ 110us) server_negotiation.cc:407] Connection header received
0808 23:04:06.484693 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:06.484693 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:06.484699 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:06.484761 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:06.485277 (+ 516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:06.485278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:06.485278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:04:06.485282 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:06.485297 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:04:06.485302 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:06.485351 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:06.485434 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 23:04:06.485448 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 23:04:06.496754 25660 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:04:06.496928 25660 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:04:06.502475 25660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74f411bd05ed44ef855912e9999374b3: Generated new TSK 0
I20260808 23:04:06.502650 25660 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:04:06.506438 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 25526
2026-08-08T23:04:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1412 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:04:06 dist-test-slave-f0tn krb5kdc[25692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:04:06 dist-test-slave-f0tn krb5kdc[25692](info): set up 2 sockets
Aug 08 23:04:06 dist-test-slave-f0tn krb5kdc[25692](info): commencing operation
krb5kdc: starting...
W20260808 23:04:07.668170 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:04:07 dist-test-slave-f0tn krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:04:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:04:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:04:07.783373 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:45473
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:46327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:45473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:04:07.867547 25713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:04:07.867734 25713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:04:07.867755 25713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:04:07.868865 25713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:04:07.868904 25713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:04:07.868917 25713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:04:07.868942 25713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:04:07.868958 25713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:04:07.870504 25713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:46327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:45473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:45473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:04:07.870726 25713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:04:07.870919 25713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:04:07.875509 25723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:07.875537 25721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:04:07.875594 25713 server_base.cc:1034] running on GCE node
W20260808 23:04:07.875560 25720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:04:07.875985 25713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:04:07.876281 25713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:04:07.877434 25713 hybrid_clock.cc:648] HybridClock initialized: now 1786230247877414 us; error 32 us; skew 500 ppm
Aug 08 23:04:07 dist-test-slave-f0tn krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230247, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:04:07.879722 25713 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:04:07.880112 25713 webserver.cc:460] Webserver started at http://127.23.194.62:36621/ using document root <none> and password file <none>
I20260808 23:04:07.880311 25713 fs_manager.cc:359] Metadata directory not provided
I20260808 23:04:07.880357 25713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:04:07.880477 25713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:04:07.881510 25713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "b432868047fa44b59d57c6a01d57bd9f"
format_stamp: "Formatted at 2026-08-08 23:04:07 on dist-test-slave-f0tn"
server_key: "a73342a4d7a245bd7da5cf37af6f9d11"
server_key_iv: "3dbc96b28bbb10e32800bdfea5ed0ade"
server_key_version: "encryptionkey@0"
I20260808 23:04:07.881841 25713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "b432868047fa44b59d57c6a01d57bd9f"
format_stamp: "Formatted at 2026-08-08 23:04:07 on dist-test-slave-f0tn"
server_key: "a73342a4d7a245bd7da5cf37af6f9d11"
server_key_iv: "3dbc96b28bbb10e32800bdfea5ed0ade"
server_key_version: "encryptionkey@0"
I20260808 23:04:07.883011 25713 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 23:04:07.883641 25739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:04:07.883785 25713 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:04:07.883867 25713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "b432868047fa44b59d57c6a01d57bd9f"
format_stamp: "Formatted at 2026-08-08 23:04:07 on dist-test-slave-f0tn"
server_key: "a73342a4d7a245bd7da5cf37af6f9d11"
server_key_iv: "3dbc96b28bbb10e32800bdfea5ed0ade"
server_key_version: "encryptionkey@0"
I20260808 23:04:07.883922 25713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:04:07.891108 25713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:04:07.892717 25713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:04:07.892838 25713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:04:07.896668 25713 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:45473
I20260808 23:04:07.896698 25820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:45473 every 8 connection(s)
I20260808 23:04:07.896982 25713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:04:07.897600 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 25713
I20260808 23:04:07.897716 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:04:07.897848 25821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:04:07.897872 24328 external_mini_cluster.cc:1428] Setting key 8d19688efd886f97578fe51d8545b73b
I20260808 23:04:07.900911 25821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f: Bootstrap starting.
I20260808 23:04:07.901626 25821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f: Neither blocks nor log segments found. Creating new log.
I20260808 23:04:07.901937 25821 log.cc:824] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f: Log is configured to *not* fsync() on all Append() calls
I20260808 23:04:07.902616 25821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f: No bootstrap required, opened a new log
Aug 08 23:04:07 dist-test-slave-f0tn krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:04:07.904943 25821 raft_consensus.cc:348] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b432868047fa44b59d57c6a01d57bd9f" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 45473 } }
I20260808 23:04:07.905102 25821 raft_consensus.cc:374] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:04:07.905128 25821 raft_consensus.cc:729] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b432868047fa44b59d57c6a01d57bd9f, State: Initialized, Role: FOLLOWER
I20260808 23:04:07.905242 25821 consensus_queue.cc:260] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b432868047fa44b59d57c6a01d57bd9f" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 45473 } }
I20260808 23:04:07.905323 25821 raft_consensus.cc:388] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:04:07.905354 25821 raft_consensus.cc:482] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:04:07.905421 25821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:04:07.906013 25821 raft_consensus.cc:504] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b432868047fa44b59d57c6a01d57bd9f" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 45473 } }
I20260808 23:04:07.906126 25821 leader_election.cc:302] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b432868047fa44b59d57c6a01d57bd9f; no voters:
I20260808 23:04:07.906276 25821 leader_election.cc:288] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:04:07.906317 25828 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:04:07.906461 25828 raft_consensus.cc:686] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f [term 1 LEADER]: Becoming Leader. State: Replica: b432868047fa44b59d57c6a01d57bd9f, State: Running, Role: LEADER
I20260808 23:04:07.906534 25821 sys_catalog.cc:564] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:04:07.906579 25828 consensus_queue.cc:237] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b432868047fa44b59d57c6a01d57bd9f" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 45473 } }
I20260808 23:04:07.906889 25828 sys_catalog.cc:455] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b432868047fa44b59d57c6a01d57bd9f. Latest consensus state: current_term: 1 leader_uuid: "b432868047fa44b59d57c6a01d57bd9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b432868047fa44b59d57c6a01d57bd9f" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 45473 } } }
I20260808 23:04:07.907092 25828 sys_catalog.cc:458] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f [sys.catalog]: This master's current role is: LEADER
I20260808 23:04:07.907006 25829 sys_catalog.cc:455] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b432868047fa44b59d57c6a01d57bd9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b432868047fa44b59d57c6a01d57bd9f" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 45473 } } }
I20260808 23:04:07.907334 25829 sys_catalog.cc:458] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f [sys.catalog]: This master's current role is: LEADER
I20260808 23:04:07.907331 25838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:04:07.907980 25838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:04:07.908762 25826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:07.898322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59485 (local address 127.23.194.62:45473)
0808 23:04:07.898493 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:07.898502 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:07.898535 (+ 33us) server_negotiation.cc:407] Connection header received
0808 23:04:07.898905 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:07.898914 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:07.898975 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:07.899120 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:07.899684 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:07.900988 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:07.902066 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:07.902457 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:07.902537 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:07.905026 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:07.905029 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:07.905030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:07.905040 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:07.906798 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:07.907153 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:07.907156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:07.907157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:07.907266 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:07.907498 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:07.907499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:07.907499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:07.907782 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:07.908355 (+ 573us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:07.908424 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 23:04:07.908557 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":72,"threads_started":1}
I20260808 23:04:07.909795 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:04:07.909844 25838 catalog_manager.cc:1347] Generated new cluster ID: 6c446b8321164a8f98b1cb664d5b28f5
I20260808 23:04:07.909929 25838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:04:07.918921 25826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:07.913667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55091 (local address 127.23.194.62:45473)
0808 23:04:07.913705 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:07.913707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:07.913830 (+ 123us) server_negotiation.cc:407] Connection header received
0808 23:04:07.913880 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:07.913881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:07.913890 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:07.913935 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:07.914391 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:07.914850 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:07.915492 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:07.915710 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:07.915762 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:07.916766 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:07.916767 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:07.916768 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:07.916774 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:07.917978 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:07.918272 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:07.918272 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:07.918273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:07.918324 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:07.918480 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:07.918481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:07.918481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:07.918578 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:07.918654 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:07.918772 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 23:04:07.918837 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 23:04:07.926750 25838 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:04:07.927410 25838 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:04:07.933806 25838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b432868047fa44b59d57c6a01d57bd9f: Generated new TSK 0
I20260808 23:04:07.933971 25838 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:04:07.939291 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 25713
2026-08-08T23:04:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1428 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:04:07 dist-test-slave-f0tn krb5kdc[25872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:04:07 dist-test-slave-f0tn krb5kdc[25872](info): set up 2 sockets
Aug 08 23:04:07 dist-test-slave-f0tn krb5kdc[25872](info): commencing operation
krb5kdc: starting...
W20260808 23:04:09.089542 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:04:09 dist-test-slave-f0tn krb5kdc[25872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:04:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:04:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:04:09.201804 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:39895
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:39973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:39895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:04:09.281891 25897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:04:09.282083 25897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:04:09.282114 25897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:04:09.283228 25897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:04:09.283270 25897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:04:09.283288 25897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:04:09.283303 25897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:04:09.283318 25897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:04:09.284771 25897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:39973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:39895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:39895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:04:09.285012 25897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:04:09.285209 25897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:04:09.289727 25906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:09.289788 25909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:09.289716 25904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:04:09.289842 25897 server_base.cc:1034] running on GCE node
I20260808 23:04:09.290184 25897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:04:09.290421 25897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:04:09.291551 25897 hybrid_clock.cc:648] HybridClock initialized: now 1786230249291533 us; error 34 us; skew 500 ppm
Aug 08 23:04:09 dist-test-slave-f0tn krb5kdc[25872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230249, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:04:09.297627 25897 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:04:09.298069 25897 webserver.cc:460] Webserver started at http://127.23.194.62:35601/ using document root <none> and password file <none>
I20260808 23:04:09.298297 25897 fs_manager.cc:359] Metadata directory not provided
I20260808 23:04:09.298344 25897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:04:09.298466 25897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:04:09.299437 25897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "dbe851d33f864509bea4f09fada8cf44"
format_stamp: "Formatted at 2026-08-08 23:04:09 on dist-test-slave-f0tn"
server_key: "e9fcfd38541689b748b4923c6dbfe26a"
server_key_iv: "63dff4d4d1f904fa262e71900029426a"
server_key_version: "encryptionkey@0"
I20260808 23:04:09.299752 25897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "dbe851d33f864509bea4f09fada8cf44"
format_stamp: "Formatted at 2026-08-08 23:04:09 on dist-test-slave-f0tn"
server_key: "e9fcfd38541689b748b4923c6dbfe26a"
server_key_iv: "63dff4d4d1f904fa262e71900029426a"
server_key_version: "encryptionkey@0"
I20260808 23:04:09.300933 25897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:04:09.301626 25917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:04:09.301781 25897 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:04:09.301863 25897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "dbe851d33f864509bea4f09fada8cf44"
format_stamp: "Formatted at 2026-08-08 23:04:09 on dist-test-slave-f0tn"
server_key: "e9fcfd38541689b748b4923c6dbfe26a"
server_key_iv: "63dff4d4d1f904fa262e71900029426a"
server_key_version: "encryptionkey@0"
I20260808 23:04:09.301927 25897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:04:09.318472 25897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:04:09.320572 25897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:04:09.320709 25897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:04:09.324512 25897 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:39895
I20260808 23:04:09.324544 25990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:39895 every 8 connection(s)
I20260808 23:04:09.324863 25897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:04:09.325726 25991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:04:09.326025 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 25897
I20260808 23:04:09.326152 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:04:09.326323 24328 external_mini_cluster.cc:1428] Setting key c3d6d7127e3ca39d629eb8164795c840
I20260808 23:04:09.328768 25991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44: Bootstrap starting.
I20260808 23:04:09.329494 25991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44: Neither blocks nor log segments found. Creating new log.
I20260808 23:04:09.329788 25991 log.cc:824] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44: Log is configured to *not* fsync() on all Append() calls
I20260808 23:04:09.330505 25991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44: No bootstrap required, opened a new log
I20260808 23:04:09.332490 25994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:09.326839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33649 (local address 127.23.194.62:39895)
0808 23:04:09.327049 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:09.327058 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:09.327093 (+ 35us) server_negotiation.cc:407] Connection header received
0808 23:04:09.327604 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:09.327613 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:09.327673 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:09.327824 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:09.328039 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:09.329355 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:09.330306 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:09.330875 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:09.330995 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:09.331804 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:09.331809 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:09.331810 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:04:09.332011 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:09.332052 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:04:09.332068 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:09.332142 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:09.332200 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 23:04:09.332272 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":91,"threads_started":1}
I20260808 23:04:09.333000 25991 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe851d33f864509bea4f09fada8cf44" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39895 } }
I20260808 23:04:09.333150 25991 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:04:09.333173 25991 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe851d33f864509bea4f09fada8cf44, State: Initialized, Role: FOLLOWER
I20260808 23:04:09.333276 25991 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe851d33f864509bea4f09fada8cf44" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39895 } }
I20260808 23:04:09.333354 25991 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:04:09.333428 25991 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:04:09.333480 25991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:04:09.333491 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:04:09.334009 25991 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe851d33f864509bea4f09fada8cf44" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39895 } }
I20260808 23:04:09.334120 25991 leader_election.cc:302] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbe851d33f864509bea4f09fada8cf44; no voters:
I20260808 23:04:09.334276 25991 leader_election.cc:288] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:04:09.334321 25996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:04:09.334446 25996 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44 [term 1 LEADER]: Becoming Leader. State: Replica: dbe851d33f864509bea4f09fada8cf44, State: Running, Role: LEADER
I20260808 23:04:09.334556 25991 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:04:09.334556 25996 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe851d33f864509bea4f09fada8cf44" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39895 } }
I20260808 23:04:09.335043 26006 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbe851d33f864509bea4f09fada8cf44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe851d33f864509bea4f09fada8cf44" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39895 } } }
I20260808 23:04:09.335188 26006 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44 [sys.catalog]: This master's current role is: LEADER
I20260808 23:04:09.335027 25996 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbe851d33f864509bea4f09fada8cf44. Latest consensus state: current_term: 1 leader_uuid: "dbe851d33f864509bea4f09fada8cf44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe851d33f864509bea4f09fada8cf44" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39895 } } }
I20260808 23:04:09.335608 25996 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44 [sys.catalog]: This master's current role is: LEADER
I20260808 23:04:09.335835 26018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:04:09.336352 26018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:04:09.337734 26018 catalog_manager.cc:1347] Generated new cluster ID: e9fc07c5cd014a2e8970b26c9f8cdf7f
I20260808 23:04:09.337779 26018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:04:09.338300 25994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:09.334804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59483 (local address 127.23.194.62:39895)
0808 23:04:09.334839 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:09.334840 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:09.335057 (+ 217us) server_negotiation.cc:407] Connection header received
0808 23:04:09.335163 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:09.335164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:09.335173 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:09.335201 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:09.335417 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:09.336093 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:09.337030 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:09.337267 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:09.337334 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:09.338059 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:09.338060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:09.338060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:04:09.338065 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:09.338080 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:04:09.338084 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:09.338110 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:09.338210 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 23:04:09.338233 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 23:04:09.342083 26018 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:04:09.342677 26018 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:04:09.351464 26018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbe851d33f864509bea4f09fada8cf44: Generated new TSK 0
I20260808 23:04:09.351663 26018 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:04:09.359279 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 25897
2026-08-08T23:04:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1422 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:04:09 dist-test-slave-f0tn krb5kdc[26037](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:04:09 dist-test-slave-f0tn krb5kdc[26037](info): set up 2 sockets
Aug 08 23:04:09 dist-test-slave-f0tn krb5kdc[26037](info): commencing operation
krb5kdc: starting...
W20260808 23:04:10.488613 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:04:10 dist-test-slave-f0tn krb5kdc[26037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:04:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:04:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:04:10.601999 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:43099
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:43099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:04:10.682086 26056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:04:10.682283 26056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:04:10.682304 26056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:04:10.683382 26056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:04:10.683414 26056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:04:10.683426 26056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:04:10.683437 26056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:04:10.683449 26056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:04:10.684857 26056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:43099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:43099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:04:10.685056 26056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:04:10.685228 26056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:04:10.689869 26066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:04:10.689971 26056 server_base.cc:1034] running on GCE node
W20260808 23:04:10.689896 26073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:10.689859 26065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:04:10.690265 26056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:04:10.690536 26056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:04:10.691686 26056 hybrid_clock.cc:648] HybridClock initialized: now 1786230250691674 us; error 28 us; skew 500 ppm
Aug 08 23:04:10 dist-test-slave-f0tn krb5kdc[26037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230250, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:04:10.694173 26056 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:04:10.694571 26056 webserver.cc:460] Webserver started at http://127.23.194.62:33029/ using document root <none> and password file <none>
I20260808 23:04:10.694770 26056 fs_manager.cc:359] Metadata directory not provided
I20260808 23:04:10.694818 26056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:04:10.694939 26056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:04:10.695969 26056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "3f8f4d3e4bd044bb825b80e875e7a502"
format_stamp: "Formatted at 2026-08-08 23:04:10 on dist-test-slave-f0tn"
server_key: "5947e4d86c21baebe23a40b4d767df47"
server_key_iv: "c52ccfced80436365f164b9c0d7e2082"
server_key_version: "encryptionkey@0"
I20260808 23:04:10.696290 26056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "3f8f4d3e4bd044bb825b80e875e7a502"
format_stamp: "Formatted at 2026-08-08 23:04:10 on dist-test-slave-f0tn"
server_key: "5947e4d86c21baebe23a40b4d767df47"
server_key_iv: "c52ccfced80436365f164b9c0d7e2082"
server_key_version: "encryptionkey@0"
I20260808 23:04:10.697536 26056 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 23:04:10.698186 26087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:04:10.698333 26056 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 23:04:10.698417 26056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "3f8f4d3e4bd044bb825b80e875e7a502"
format_stamp: "Formatted at 2026-08-08 23:04:10 on dist-test-slave-f0tn"
server_key: "5947e4d86c21baebe23a40b4d767df47"
server_key_iv: "c52ccfced80436365f164b9c0d7e2082"
server_key_version: "encryptionkey@0"
I20260808 23:04:10.698478 26056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:04:10.701630 26056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:04:10.703251 26056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:04:10.703370 26056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:04:10.707031 26056 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:43099
I20260808 23:04:10.707065 26168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:43099 every 8 connection(s)
I20260808 23:04:10.707331 26056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:04:10.708227 26169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:04:10.710390 26169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502: Bootstrap starting.
I20260808 23:04:10.710917 26169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502: Neither blocks nor log segments found. Creating new log.
I20260808 23:04:10.711174 26169 log.cc:824] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502: Log is configured to *not* fsync() on all Append() calls
I20260808 23:04:10.711853 26169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502: No bootstrap required, opened a new log
I20260808 23:04:10.713340 26169 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f8f4d3e4bd044bb825b80e875e7a502" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 43099 } }
I20260808 23:04:10.713506 26169 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:04:10.713546 26169 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f8f4d3e4bd044bb825b80e875e7a502, State: Initialized, Role: FOLLOWER
I20260808 23:04:10.713687 26169 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f8f4d3e4bd044bb825b80e875e7a502" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 43099 } }
I20260808 23:04:10.713748 26169 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:04:10.713770 26169 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:04:10.713816 26169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:04:10.714582 26169 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f8f4d3e4bd044bb825b80e875e7a502" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 43099 } }
I20260808 23:04:10.714679 26169 leader_election.cc:302] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f8f4d3e4bd044bb825b80e875e7a502; no voters:
I20260808 23:04:10.714824 26169 leader_election.cc:288] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:04:10.714847 26173 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:04:10.714991 26173 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502 [term 1 LEADER]: Becoming Leader. State: Replica: 3f8f4d3e4bd044bb825b80e875e7a502, State: Running, Role: LEADER
I20260808 23:04:10.715049 26169 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:04:10.715082 26173 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f8f4d3e4bd044bb825b80e875e7a502" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 43099 } }
I20260808 23:04:10.715418 26178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f8f4d3e4bd044bb825b80e875e7a502. Latest consensus state: current_term: 1 leader_uuid: "3f8f4d3e4bd044bb825b80e875e7a502" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f8f4d3e4bd044bb825b80e875e7a502" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 43099 } } }
I20260808 23:04:10.715529 26178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502 [sys.catalog]: This master's current role is: LEADER
I20260808 23:04:10.715407 26175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f8f4d3e4bd044bb825b80e875e7a502" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f8f4d3e4bd044bb825b80e875e7a502" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 43099 } } }
I20260808 23:04:10.715744 26175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502 [sys.catalog]: This master's current role is: LEADER
I20260808 23:04:10.715776 26185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:04:10.716248 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 26056
I20260808 23:04:10.716302 26185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:04:10.716368 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:04:10.716552 24328 external_mini_cluster.cc:1428] Setting key 736dcef2460b90c1c8106a9efd4df56d
I20260808 23:04:10.717854 26185 catalog_manager.cc:1347] Generated new cluster ID: 5915c35be12d45bcae5182043414003b
I20260808 23:04:10.717906 26185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:04:10.726100 26207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:10.716947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55545 (local address 127.23.194.62:43099)
0808 23:04:10.717317 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:10.717326 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:10.717358 (+ 32us) server_negotiation.cc:407] Connection header received
0808 23:04:10.717571 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:10.717578 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:10.717607 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:10.717753 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:10.725654 (+ 7901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:10.725657 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:10.725657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:04:10.725662 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:10.725693 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:04:10.725706 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:10.725770 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:10.725866 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 23:04:10.725904 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":73,"threads_started":1}
I20260808 23:04:10.726974 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:04:10.727855 26185 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:04:10.727993 26185 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:04:10.729465 26207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:10.728470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47951 (local address 127.23.194.62:43099)
0808 23:04:10.728499 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:10.728500 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:10.728636 (+ 136us) server_negotiation.cc:407] Connection header received
0808 23:04:10.728705 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:10.728706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:10.728710 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:10.728763 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:10.729266 (+ 503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:10.729267 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:10.729267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:04:10.729271 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:10.729284 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:04:10.729288 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:10.729333 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:10.729395 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 23:04:10.729407 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 23:04:10.743814 26185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f8f4d3e4bd044bb825b80e875e7a502: Generated new TSK 0
I20260808 23:04:10.743996 26185 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:04:10.747834 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 26056
2026-08-08T23:04:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1384 ms)
[----------] 18 tests from AuthTokenIssuingTest (25452 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:04:10 dist-test-slave-f0tn krb5kdc[26220](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:04:10 dist-test-slave-f0tn krb5kdc[26220](info): set up 2 sockets
Aug 08 23:04:10 dist-test-slave-f0tn krb5kdc[26220](info): commencing operation
krb5kdc: starting...
W20260808 23:04:11.879356 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:04:11 dist-test-slave-f0tn krb5kdc[26220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:04:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:04:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:04:11.989128 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:39523
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:39523,127.23.194.61:34183,127.23.194.60:44673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:04:12.068729 26238 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 23:04:12.068931 26238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:04:12.068955 26238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:04:12.068979 26238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:04:12.070104 26238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:04:12.070143 26238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:04:12.070161 26238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:04:12.070178 26238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:04:12.070191 26238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:04:12.071702 26238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:39523,127.23.194.61:34183,127.23.194.60:44673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:39523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:04:12.071940 26238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:04:12.072141 26238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:04:12.076819 26246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:12.076829 26247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:12.076860 26251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:04:12.077050 26238 server_base.cc:1034] running on GCE node
I20260808 23:04:12.077286 26238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:04:12.077620 26238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:04:12.078778 26238 hybrid_clock.cc:648] HybridClock initialized: now 1786230252078766 us; error 31 us; skew 500 ppm
Aug 08 23:04:12 dist-test-slave-f0tn krb5kdc[26220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230252, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:04:12.081163 26238 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:04:12.081612 26238 webserver.cc:460] Webserver started at http://127.23.194.62:38251/ using document root <none> and password file <none>
I20260808 23:04:12.081813 26238 fs_manager.cc:359] Metadata directory not provided
I20260808 23:04:12.081856 26238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:04:12.081947 26238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:04:12.082926 26238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "ce19ea795cc849b599b5e3a69365f711"
format_stamp: "Formatted at 2026-08-08 23:04:12 on dist-test-slave-f0tn"
server_key: "f3beddfdf0fa8f8d8a1c5f855b7a7e6e"
server_key_iv: "719148621bbdb08d226be55412d5a2b0"
server_key_version: "encryptionkey@0"
I20260808 23:04:12.083211 26238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "ce19ea795cc849b599b5e3a69365f711"
format_stamp: "Formatted at 2026-08-08 23:04:12 on dist-test-slave-f0tn"
server_key: "f3beddfdf0fa8f8d8a1c5f855b7a7e6e"
server_key_iv: "719148621bbdb08d226be55412d5a2b0"
server_key_version: "encryptionkey@0"
I20260808 23:04:12.084371 26238 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:04:12.085016 26260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:04:12.085183 26238 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:04:12.085270 26238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "ce19ea795cc849b599b5e3a69365f711"
format_stamp: "Formatted at 2026-08-08 23:04:12 on dist-test-slave-f0tn"
server_key: "f3beddfdf0fa8f8d8a1c5f855b7a7e6e"
server_key_iv: "719148621bbdb08d226be55412d5a2b0"
server_key_version: "encryptionkey@0"
I20260808 23:04:12.085323 26238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:04:12.103806 26238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:04:12.105861 26238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:04:12.105968 26238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:04:12.109659 26238 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:39523
I20260808 23:04:12.109694 26353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:39523 every 8 connection(s)
I20260808 23:04:12.109999 26238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:04:12.110961 26354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:04:12.112532 26354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:04:12.113306 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 26238
I20260808 23:04:12.113443 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:04:12.113632 24328 external_mini_cluster.cc:1428] Setting key d994f7d7dad0a5a7a03675af71505444
Aug 08 23:04:12 dist-test-slave-f0tn krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230252, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.194.61@KRBTEST.COM; defaulting to no policy
I20260808 23:04:12.121949 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.113132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:39523 (local address 127.0.0.1:41644)
0808 23:04:12.113307 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.113353 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 23:04:12.113960 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:04:12.114235 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:04:12.114237 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:04:12.114550 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:04:12.114773 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:12.114779 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.115736 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.115737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:12.116379 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:12.116381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.116850 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.116851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:12.116925 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.117352 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:04:12.117360 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:04:12.118895 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:04:12.120443 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:12.120450 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:12.120453 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:12.120715 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:12.120924 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:12.120925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:12.120925 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:12.121031 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:12.121338 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:04:12.121345 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:04:12.121440 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 23:04:12.121541 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 23:04:12.121708 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260808 23:04:12.122077 26358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.113298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41644 (local address 127.23.194.62:39523)
0808 23:04:12.113441 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.113446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.113483 (+ 37us) server_negotiation.cc:407] Connection header received
0808 23:04:12.114104 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.114106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.114123 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.114202 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.114854 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.115649 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.116460 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.116761 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.116854 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.118977 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:12.118984 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:12.118985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:12.118991 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:12.120361 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.120787 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.120789 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.120790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.120860 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.121090 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.121090 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.121091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.121279 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:12.121335 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.121870 (+ 535us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.121941 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":42,"threads_started":1}
I20260808 23:04:12.122685 26354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:04:12.122964 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.122867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:34183 (local address 127.0.0.1:43334)
0808 23:04:12.122904 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.122945 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.61:34183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 23:04:12.123082 26266 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.194.61:34183: connect: Connection refused (error 111)
I20260808 23:04:12.123436 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.123345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:34183 (local address 127.0.0.1:43342)
0808 23:04:12.123388 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.123416 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.61:34183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 23:04:12.123576 26354 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.61:34183: Network error: Client connection negotiation failed: client connection to 127.23.194.61:34183: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.61.keytab.
Entry for principal kudu/127.23.194.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.61.keytab.
I20260808 23:04:12.136907 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.61:34183
--webserver_interface=127.23.194.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.61.keytab
--principal=kudu/127.23.194.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:39523,127.23.194.61:34183,127.23.194.60:44673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:04:12.172798 26354 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } attempt: 1
I20260808 23:04:12.173302 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.173169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:34183 (local address 127.0.0.1:43348)
0808 23:04:12.173227 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.173269 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.61:34183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 23:04:12.173655 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.173562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:34183 (local address 127.0.0.1:43350)
0808 23:04:12.173596 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.173620 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.61:34183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 23:04:12.173738 26354 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.61:34183: Network error: Client connection negotiation failed: client connection to 127.23.194.61:34183: connect: Connection refused (error 111)
W20260808 23:04:12.217226 26366 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 23:04:12.217458 26366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:04:12.217475 26366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:04:12.217497 26366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:04:12.218642 26366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:04:12.218684 26366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:04:12.218698 26366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:04:12.218720 26366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:04:12.218742 26366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:04:12.220371 26366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:39523,127.23.194.61:34183,127.23.194.60:44673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.61.keytab
--principal=kudu/127.23.194.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.61:34183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.194.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:04:12.220634 26366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:04:12.220890 26366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:04:12.225564 26376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:12.225541 26379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:12.225540 26374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:04:12.225565 26366 server_base.cc:1034] running on GCE node
I20260808 23:04:12.225961 26366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:04:12.226235 26366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:04:12.227380 26366 hybrid_clock.cc:648] HybridClock initialized: now 1786230252227365 us; error 27 us; skew 500 ppm
Aug 08 23:04:12 dist-test-slave-f0tn krb5kdc[26220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230252, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:04:12.229816 26366 init.cc:377] Logged in from keytab as kudu/127.23.194.61@KRBTEST.COM (short username kudu)
I20260808 23:04:12.230194 26366 webserver.cc:460] Webserver started at http://127.23.194.61:43693/ using document root <none> and password file <none>
I20260808 23:04:12.230388 26366 fs_manager.cc:359] Metadata directory not provided
I20260808 23:04:12.230427 26366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:04:12.230515 26366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:04:12.231479 26366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-1/data/instance:
uuid: "c871319bdb7a44cba7c0d6d0f8bb8293"
format_stamp: "Formatted at 2026-08-08 23:04:12 on dist-test-slave-f0tn"
server_key: "d47bf9a17829e48fee13a53ca04bd43a"
server_key_iv: "bd0df44953915c1941c0e212f64b536d"
server_key_version: "encryptionkey@0"
I20260808 23:04:12.231762 26366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-1/wal/instance:
uuid: "c871319bdb7a44cba7c0d6d0f8bb8293"
format_stamp: "Formatted at 2026-08-08 23:04:12 on dist-test-slave-f0tn"
server_key: "d47bf9a17829e48fee13a53ca04bd43a"
server_key_iv: "bd0df44953915c1941c0e212f64b536d"
server_key_version: "encryptionkey@0"
I20260808 23:04:12.232899 26366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 23:04:12.233523 26390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:04:12.233673 26366 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:04:12.233758 26366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-1/wal
uuid: "c871319bdb7a44cba7c0d6d0f8bb8293"
format_stamp: "Formatted at 2026-08-08 23:04:12 on dist-test-slave-f0tn"
server_key: "d47bf9a17829e48fee13a53ca04bd43a"
server_key_iv: "bd0df44953915c1941c0e212f64b536d"
server_key_version: "encryptionkey@0"
I20260808 23:04:12.233819 26366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:04:12.241860 26354 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } attempt: 2
I20260808 23:04:12.242221 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.242083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:34183 (local address 127.0.0.1:43366)
0808 23:04:12.242143 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.242184 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.61:34183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 23:04:12.242540 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.242437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:34183 (local address 127.0.0.1:43372)
0808 23:04:12.242485 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.242516 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.61:34183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 23:04:12.242631 26354 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.61:34183: Network error: Client connection negotiation failed: client connection to 127.23.194.61:34183: connect: Connection refused (error 111)
I20260808 23:04:12.243402 26366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:04:12.245291 26366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:04:12.245447 26366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:04:12.249028 26366 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.61:34183
I20260808 23:04:12.249073 26471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.61:34183 every 8 connection(s)
I20260808 23:04:12.249353 26366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-1/data/info.pb
I20260808 23:04:12.250288 26472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:04:12.251183 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 26366
I20260808 23:04:12.251283 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-1/wal/instance
I20260808 23:04:12.251451 24328 external_mini_cluster.cc:1428] Setting key fe51d38b5203cea5c4398f168a61fe10
I20260808 23:04:12.252043 26472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:04:12 dist-test-slave-f0tn krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230252, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.61@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.194.60@KRBTEST.COM; defaulting to no policy
I20260808 23:04:12.260246 26358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.252735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41660 (local address 127.23.194.62:39523)
0808 23:04:12.252778 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.252780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.252995 (+ 215us) server_negotiation.cc:407] Connection header received
0808 23:04:12.253574 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.253575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.253590 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.253640 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.254202 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.254679 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.255660 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.255863 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.255917 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.257951 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:12.257952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:12.257952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:12.257959 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:12.259216 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.259563 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.259564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.259564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.259610 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.259765 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.259765 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.259765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.259855 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:12.259899 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.260118 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.260186 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 23:04:12.260362 26477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.252709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:39523 (local address 127.0.0.1:41660)
0808 23:04:12.252866 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.252910 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 23:04:12.253480 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:04:12.253703 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:04:12.253707 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:04:12.253929 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:04:12.254115 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:12.254118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.254757 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.254759 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:12.255576 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:12.255579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.255942 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.255943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:12.256058 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.256474 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:04:12.256482 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:04:12.257862 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:04:12.259288 (+ 1426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:12.259296 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:12.259298 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:12.259503 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:12.259646 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:12.259646 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:12.259647 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:12.259727 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:12.259890 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:04:12.259892 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:04:12.260014 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 23:04:12.260091 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 23:04:12.260201 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
I20260808 23:04:12.260869 26472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.60@KRBTEST.COM" created.
Aug 08 23:04:12 dist-test-slave-f0tn krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230252, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.61@KRBTEST.COM for kudu/127.23.194.61@KRBTEST.COM
I20260808 23:04:12.267382 26477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.261098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:34183 (local address 127.0.0.1:43382)
0808 23:04:12.261128 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.261138 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:04:12.261186 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:04:12.261447 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:04:12.261448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:04:12.261575 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:04:12.261622 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:12.261624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.262440 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.262441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:12.262911 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:12.262912 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.263436 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.263437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:12.263507 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.263819 (+ 312us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:04:12.263825 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:04:12.264961 (+ 1136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:04:12.266486 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:12.266487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:12.266487 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:12.266633 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:12.266801 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:12.266802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:12.266802 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:12.266854 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:12.267086 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:04:12.267087 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:04:12.267122 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 23:04:12.267192 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 23:04:12.267284 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260808 23:04:12.267453 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.261202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43382 (local address 127.23.194.61:34183)
0808 23:04:12.261321 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.261325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.261339 (+ 14us) server_negotiation.cc:407] Connection header received
0808 23:04:12.261354 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.261355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.261365 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.261448 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.261680 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.262363 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.263000 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.263366 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.263421 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.265035 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:12.265043 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:12.265043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:12.265050 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:12.266413 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.266700 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.266703 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.266703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.266749 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.266904 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.266904 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.266905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.267026 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:12.267093 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.267226 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.267337 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
I20260808 23:04:12.267987 26472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:04:12.268296 26477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.268193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:44673 (local address 127.0.0.1:41982)
0808 23:04:12.268232 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.268274 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:44673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 23:04:12.268414 26392 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.194.60:44673: connect: Connection refused (error 111)
I20260808 23:04:12.268780 26477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.268700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:44673 (local address 127.0.0.1:41992)
0808 23:04:12.268734 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.268758 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:44673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 23:04:12.268875 26472 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.60:44673: Network error: Client connection negotiation failed: client connection to 127.23.194.60:44673: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.60.keytab.
Entry for principal kudu/127.23.194.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.60.keytab.
I20260808 23:04:12.276425 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.60:44673
--webserver_interface=127.23.194.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.60.keytab
--principal=kudu/127.23.194.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:37773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:39523,127.23.194.61:34183,127.23.194.60:44673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:04:12.318075 26472 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } attempt: 1
I20260808 23:04:12.318573 26477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.318442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:44673 (local address 127.0.0.1:42006)
0808 23:04:12.318501 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.318543 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:44673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 23:04:12.318887 26477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.318817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:44673 (local address 127.0.0.1:42018)
0808 23:04:12.318849 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.318871 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:44673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260808 23:04:12.318987 26472 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.60:44673: Network error: Client connection negotiation failed: client connection to 127.23.194.60:44673: connect: Connection refused (error 111)
I20260808 23:04:12.333791 26354 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } attempt: 3
Aug 08 23:04:12 dist-test-slave-f0tn krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230252, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for kudu/127.23.194.61@KRBTEST.COM
I20260808 23:04:12.340845 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.334251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43398 (local address 127.23.194.61:34183)
0808 23:04:12.334310 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.334312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.334345 (+ 33us) server_negotiation.cc:407] Connection header received
0808 23:04:12.334433 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.334434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.334449 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.334502 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.334854 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.335374 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.336175 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.336403 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.336469 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.338463 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:12.338464 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:12.338465 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:12.338473 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:12.339828 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.340145 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.340146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.340146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.340199 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.340356 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.340357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.340358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.340461 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:12.340515 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.340674 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.340746 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 23:04:12.340847 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.334205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:34183 (local address 127.0.0.1:43398)
0808 23:04:12.334274 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.334286 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 23:04:12.334366 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:04:12.334525 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:04:12.334527 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:04:12.334698 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:04:12.334773 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:12.334776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.335488 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.335490 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:12.336096 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:12.336099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.336473 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.336483 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:12.336567 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.337038 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:04:12.337043 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:04:12.338372 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:04:12.339914 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:12.339915 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:12.339915 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:12.340082 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:12.340252 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:12.340253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:12.340253 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:12.340309 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:12.340514 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:04:12.340516 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:04:12.340588 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 23:04:12.340644 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 23:04:12.340746 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 23:04:12.341259 26354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:04:12.341603 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.341498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:44673 (local address 127.0.0.1:42022)
0808 23:04:12.341541 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.341575 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:44673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 23:04:12.341917 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.341836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:44673 (local address 127.0.0.1:42028)
0808 23:04:12.341874 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.341896 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:44673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 23:04:12.341996 26354 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.60:44673: Network error: Client connection negotiation failed: client connection to 127.23.194.60:44673: connect: Connection refused (error 111)
W20260808 23:04:12.358604 26486 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 23:04:12.358790 26486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:04:12.358814 26486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:04:12.358839 26486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:04:12.359917 26486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:04:12.359956 26486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:04:12.359974 26486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:04:12.359989 26486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:04:12.360006 26486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:04:12.361521 26486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:37773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:39523,127.23.194.61:34183,127.23.194.60:44673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.60.keytab
--principal=kudu/127.23.194.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.60:44673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.194.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:04:12.361769 26486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:04:12.361967 26486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:04:12.366482 26494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:12.366510 26497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:04:12.366523 26486 server_base.cc:1034] running on GCE node
W20260808 23:04:12.366488 26495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:04:12.366932 26486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:04:12.367213 26486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:04:12.368352 26486 hybrid_clock.cc:648] HybridClock initialized: now 1786230252368341 us; error 24 us; skew 500 ppm
Aug 08 23:04:12 dist-test-slave-f0tn krb5kdc[26220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230252, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:04:12.370621 26486 init.cc:377] Logged in from keytab as kudu/127.23.194.60@KRBTEST.COM (short username kudu)
I20260808 23:04:12.371004 26486 webserver.cc:460] Webserver started at http://127.23.194.60:37107/ using document root <none> and password file <none>
I20260808 23:04:12.371249 26486 fs_manager.cc:359] Metadata directory not provided
I20260808 23:04:12.371311 26486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:04:12.371444 26486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:04:12.372412 26486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-2/data/instance:
uuid: "2b7bac880c5341129348edf00a95dc87"
format_stamp: "Formatted at 2026-08-08 23:04:12 on dist-test-slave-f0tn"
server_key: "57c301aa514e24c7db10e136ff0585e7"
server_key_iv: "2648c906974d6d0d1a7e7580cb5f41a0"
server_key_version: "encryptionkey@0"
I20260808 23:04:12.372735 26486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-2/wal/instance:
uuid: "2b7bac880c5341129348edf00a95dc87"
format_stamp: "Formatted at 2026-08-08 23:04:12 on dist-test-slave-f0tn"
server_key: "57c301aa514e24c7db10e136ff0585e7"
server_key_iv: "2648c906974d6d0d1a7e7580cb5f41a0"
server_key_version: "encryptionkey@0"
I20260808 23:04:12.373126 26354 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } attempt: 1
I20260808 23:04:12.373502 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.373354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:44673 (local address 127.0.0.1:42034)
0808 23:04:12.373426 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.373465 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:44673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 23:04:12.373790 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.373713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:44673 (local address 127.0.0.1:42046)
0808 23:04:12.373740 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.373767 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:44673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260808 23:04:12.373874 26354 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.60:44673: Network error: Client connection negotiation failed: client connection to 127.23.194.60:44673: connect: Connection refused (error 111)
I20260808 23:04:12.374202 26486 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:04:12.374881 26506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:04:12.375020 26486 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:04:12.375098 26486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-2/wal
uuid: "2b7bac880c5341129348edf00a95dc87"
format_stamp: "Formatted at 2026-08-08 23:04:12 on dist-test-slave-f0tn"
server_key: "57c301aa514e24c7db10e136ff0585e7"
server_key_iv: "2648c906974d6d0d1a7e7580cb5f41a0"
server_key_version: "encryptionkey@0"
I20260808 23:04:12.375173 26486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:04:12.387123 26472 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } attempt: 2
I20260808 23:04:12.387519 26477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.387393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:44673 (local address 127.0.0.1:42062)
0808 23:04:12.387446 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.387485 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:44673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 23:04:12.387822 26477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.387749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:44673 (local address 127.0.0.1:42070)
0808 23:04:12.387786 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.387807 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:44673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 23:04:12.387904 26472 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.60:44673: Network error: Client connection negotiation failed: client connection to 127.23.194.60:44673: connect: Connection refused (error 111)
I20260808 23:04:12.395689 26486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:04:12.397357 26486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:04:12.397521 26486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:04:12.401082 26486 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.60:44673
I20260808 23:04:12.401120 26580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.60:44673 every 8 connection(s)
I20260808 23:04:12.401499 26486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-2/data/info.pb
I20260808 23:04:12.401966 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 26486
I20260808 23:04:12.402055 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786229981209642-24328-0/minicluster-data/master-2/wal/instance
I20260808 23:04:12.402220 24328 external_mini_cluster.cc:1428] Setting key 7de92b807b640eedf13acb1cd52fafcd
I20260808 23:04:12.402506 26581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:04:12.404660 26581 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:04:12 dist-test-slave-f0tn krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.60@KRBTEST.COM
Aug 08 23:04:12 dist-test-slave-f0tn krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230252, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.60@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:04:12.412180 26358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.405254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41674 (local address 127.23.194.62:39523)
0808 23:04:12.405306 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.405308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.405455 (+ 147us) server_negotiation.cc:407] Connection header received
0808 23:04:12.405529 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.405530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.405538 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.405581 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.406083 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.406725 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.407721 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.407900 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.407945 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.409865 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:12.409866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:12.409866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:12.409872 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:12.411081 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.411426 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.411427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.411427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.411471 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.411641 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.411642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.411642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.411747 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:12.411801 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.411990 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.412069 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 23:04:12.412266 26588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.405203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:39523 (local address 127.0.0.1:41674)
0808 23:04:12.405344 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.405407 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 23:04:12.405473 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:04:12.405609 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:04:12.405612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:04:12.405937 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:04:12.406015 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:12.406017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.406808 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.406809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:12.407632 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:12.407636 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.407990 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.407991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:12.408059 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.408470 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:04:12.408478 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:04:12.409788 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:04:12.411144 (+ 1356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:12.411150 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:12.411159 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:12.411362 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:12.411531 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:12.411532 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:12.411532 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:12.411589 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:12.411800 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:04:12.411807 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:04:12.411871 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 23:04:12.411941 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 23:04:12.412073 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":31,"thread_start_us":74,"threads_started":1}
I20260808 23:04:12.412268 26587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.402674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42082 (local address 127.23.194.60:44673)
0808 23:04:12.402838 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.402847 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.402871 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:04:12.403277 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.403283 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.403326 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.403431 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.404211 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.405173 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.406251 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.406848 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.406933 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.409255 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:12.409262 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:12.409263 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:12.409269 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:12.410888 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.411212 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.411214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.411215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.411291 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.411463 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.411464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.411464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.411678 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:12.411796 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.411889 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.412022 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":118,"thread_start_us":72,"threads_started":1}
I20260808 23:04:12.412879 26581 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:04:12 dist-test-slave-f0tn krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230252, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.60@KRBTEST.COM for kudu/127.23.194.61@KRBTEST.COM
Aug 08 23:04:12 dist-test-slave-f0tn krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:04:12.418618 26588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.413087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:34183 (local address 127.0.0.1:43406)
0808 23:04:12.413121 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.413131 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:04:12.413167 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:04:12.413324 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:04:12.413325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:04:12.413473 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:04:12.413506 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:12.413508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.414070 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.414071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:12.414530 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:12.414532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.414877 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.414878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:12.414929 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.415247 (+ 318us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:04:12.415251 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:04:12.416304 (+ 1053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:04:12.417733 (+ 1429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:12.417734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:12.417734 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:12.417905 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:12.418084 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:12.418085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:12.418086 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:12.418140 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:12.418346 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:04:12.418347 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:04:12.418386 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 23:04:12.418446 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 23:04:12.418530 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 23:04:12.418622 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.413187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43406 (local address 127.23.194.61:34183)
0808 23:04:12.413226 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.413227 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.413234 (+ 7us) server_negotiation.cc:407] Connection header received
0808 23:04:12.413257 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.413258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.413266 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.413306 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.413576 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.414004 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.414601 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.414822 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.414874 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.416383 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:12.416385 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:12.416385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:12.416392 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:12.417652 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.417975 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.417977 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.417977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.418031 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.418191 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.418192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.418192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.418294 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:12.418347 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.418475 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.418536 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 23:04:12.419183 26358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.413047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41684 (local address 127.23.194.62:39523)
0808 23:04:12.413080 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.413082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.413090 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:04:12.413132 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.413132 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.413140 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.413196 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.413767 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.414186 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.414799 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.414986 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.415036 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.416939 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:12.416940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:12.416940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:12.416946 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:12.418210 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.418488 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.418489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.418489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.418536 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.418742 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.418743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.418743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.418859 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:12.418964 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.419048 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.419101 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 23:04:12.419217 26581 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:04:12 dist-test-slave-f0tn krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230252, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.60@KRBTEST.COM for kudu/127.23.194.60@KRBTEST.COM
Aug 08 23:04:12 dist-test-slave-f0tn krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.61@KRBTEST.COM
I20260808 23:04:12.425172 26588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.419436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:44673 (local address 127.0.0.1:42086)
0808 23:04:12.419505 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.419514 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:04:12.419558 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:04:12.419661 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:04:12.419662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:04:12.419788 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:04:12.419819 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:12.419821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.420451 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.420451 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:12.420947 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:12.420949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.421260 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.421261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:12.421311 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.421639 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:04:12.421643 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:04:12.422868 (+ 1225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:04:12.424241 (+ 1373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:12.424242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:12.424242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:12.424419 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:12.424611 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:12.424612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:12.424612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:12.424669 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:12.424873 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:04:12.424874 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:04:12.424913 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 23:04:12.424977 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 23:04:12.425062 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 23:04:12.425220 26587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.419473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42086 (local address 127.23.194.60:44673)
0808 23:04:12.419506 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.419508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.419550 (+ 42us) server_negotiation.cc:407] Connection header received
0808 23:04:12.419599 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.419600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.419608 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.419656 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.419885 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.420381 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.421016 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.421206 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.421251 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.422938 (+ 1687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:12.422939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:12.422939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:12.422945 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:12.424167 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.424484 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.424485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.424485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.424548 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.424725 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.424726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.424726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.424818 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:12.424866 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.425012 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.425116 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":15}
I20260808 23:04:12.426265 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.419765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43412 (local address 127.23.194.61:34183)
0808 23:04:12.419795 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.419796 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.419841 (+ 45us) server_negotiation.cc:407] Connection header received
0808 23:04:12.419896 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.419897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.419906 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.419953 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.420318 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.420783 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.421527 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.421721 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.421782 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.423896 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:12.423897 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:12.423897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:12.423904 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:12.425269 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.425622 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.425623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.425624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.425674 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.425831 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.425832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.425832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.425926 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:12.426062 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.426105 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.426182 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 23:04:12.426646 26581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87: Bootstrap starting.
I20260808 23:04:12.427526 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:04:12.427563 26581 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87: Neither blocks nor log segments found. Creating new log.
I20260808 23:04:12.427827 26581 log.cc:824] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87: Log is configured to *not* fsync() on all Append() calls
I20260808 23:04:12.428557 26581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87: No bootstrap required, opened a new log
I20260808 23:04:12.428722 26301 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ce19ea795cc849b599b5e3a69365f711"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41684
I20260808 23:04:12.429957 26581 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce19ea795cc849b599b5e3a69365f711" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } } peers { permanent_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } } peers { permanent_uuid: "2b7bac880c5341129348edf00a95dc87" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } }
I20260808 23:04:12.430071 26581 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b7bac880c5341129348edf00a95dc87, State: Initialized, Role: FOLLOWER
I20260808 23:04:12.430153 26581 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce19ea795cc849b599b5e3a69365f711" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } } peers { permanent_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } } peers { permanent_uuid: "2b7bac880c5341129348edf00a95dc87" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } }
I20260808 23:04:12.430451 26581 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:04:12.430433 26591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce19ea795cc849b599b5e3a69365f711" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } } peers { permanent_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } } peers { permanent_uuid: "2b7bac880c5341129348edf00a95dc87" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } } }
I20260808 23:04:12.430536 26591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 23:04:12.432049 26607 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:04:12.432106 26607 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:04:12.432376 26486 master_runner.cc:186] Error getting master registration for 127.23.194.62:39523: OK, instance_id { permanent_uuid: "ce19ea795cc849b599b5e3a69365f711" instance_seqno: 1786230252108612 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 23:04:12.435048 26486 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 23:04:12.448979 26354 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } attempt: 2
Aug 08 23:04:12 dist-test-slave-f0tn krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230252, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for kudu/127.23.194.60@KRBTEST.COM
I20260808 23:04:12.455417 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.449253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:44673 (local address 127.0.0.1:42094)
0808 23:04:12.449305 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.449315 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:04:12.449353 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:04:12.449499 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:04:12.449500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:04:12.449658 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:04:12.449700 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:12.449701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.450287 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.450288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:12.450792 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:12.450794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.451111 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.451112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:12.451164 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.451562 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:04:12.451567 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:04:12.452914 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:04:12.454384 (+ 1470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:12.454384 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:12.454385 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:12.454596 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:12.454788 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:12.454788 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:12.454789 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:12.454852 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:12.455118 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:04:12.455119 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:04:12.455165 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 23:04:12.455233 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 23:04:12.455320 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 23:04:12.455418 26587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.449348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42094 (local address 127.23.194.60:44673)
0808 23:04:12.449408 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.449410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.449418 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:04:12.449439 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.449440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.449449 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.449495 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.449771 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.450221 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.450868 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.451053 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.451119 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.453013 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:12.453015 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:12.453015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:12.453022 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:12.454301 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.454675 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.454676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.454676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.454730 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.454910 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.454910 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.454911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.455063 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:12.455121 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.455257 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.455329 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 23:04:12.456940 26354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711: Bootstrap starting.
I20260808 23:04:12.457587 26354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711: Neither blocks nor log segments found. Creating new log.
I20260808 23:04:12.457854 26354 log.cc:824] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711: Log is configured to *not* fsync() on all Append() calls
I20260808 23:04:12.458524 26354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711: No bootstrap required, opened a new log
I20260808 23:04:12.459486 26354 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce19ea795cc849b599b5e3a69365f711" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } } peers { permanent_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } } peers { permanent_uuid: "2b7bac880c5341129348edf00a95dc87" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } }
I20260808 23:04:12.459591 26354 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce19ea795cc849b599b5e3a69365f711, State: Initialized, Role: FOLLOWER
I20260808 23:04:12.459664 26354 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce19ea795cc849b599b5e3a69365f711" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } } peers { permanent_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } } peers { permanent_uuid: "2b7bac880c5341129348edf00a95dc87" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } }
I20260808 23:04:12.459916 26612 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce19ea795cc849b599b5e3a69365f711" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } } peers { permanent_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } } peers { permanent_uuid: "2b7bac880c5341129348edf00a95dc87" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } } }
I20260808 23:04:12.459976 26354 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:04:12.460011 26301 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 23:04:12.460050 26612 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:04:12.460106 26301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:04:12.460928 26301 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce19ea795cc849b599b5e3a69365f711" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } } peers { permanent_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } } peers { permanent_uuid: "2b7bac880c5341129348edf00a95dc87" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } }
I20260808 23:04:12.461477 26547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce19ea795cc849b599b5e3a69365f711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2b7bac880c5341129348edf00a95dc87"
I20260808 23:04:12.461606 26547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:04:12.462422 26547 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce19ea795cc849b599b5e3a69365f711 in term 1.
I20260808 23:04:12.462543 26301 leader_election.cc:288] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [CANDIDATE]: Term 1 election: Requested vote from peers c871319bdb7a44cba7c0d6d0f8bb8293 (127.23.194.61:34183), 2b7bac880c5341129348edf00a95dc87 (127.23.194.60:44673)
I20260808 23:04:12.462675 26261 leader_election.cc:302] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b7bac880c5341129348edf00a95dc87, ce19ea795cc849b599b5e3a69365f711; no voters:
I20260808 23:04:12.462783 26612 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:04:12.462910 26612 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [term 1 LEADER]: Becoming Leader. State: Replica: ce19ea795cc849b599b5e3a69365f711, State: Running, Role: LEADER
I20260808 23:04:12.462991 26612 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce19ea795cc849b599b5e3a69365f711" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } } peers { permanent_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } } peers { permanent_uuid: "2b7bac880c5341129348edf00a95dc87" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } }
W20260808 23:04:12.463217 26635 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:04:12.463260 26635 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:04:12.463380 26612 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce19ea795cc849b599b5e3a69365f711. Latest consensus state: current_term: 1 leader_uuid: "ce19ea795cc849b599b5e3a69365f711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce19ea795cc849b599b5e3a69365f711" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } } peers { permanent_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } } peers { permanent_uuid: "2b7bac880c5341129348edf00a95dc87" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } } }
I20260808 23:04:12.463443 26612 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [sys.catalog]: This master's current role is: LEADER
I20260808 23:04:12.463737 26643 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:04:12.463948 26643 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:04:12.462438 26238 master_runner.cc:186] Error getting master registration for 127.23.194.61:34183: OK, instance_id { permanent_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293" instance_seqno: 1786230252248057 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 23:04:12.465032 26238 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 23:04:12.466635 26547 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87 [term 1 FOLLOWER]: Refusing update from remote peer ce19ea795cc849b599b5e3a69365f711: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:04:12.467641 26612 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b7bac880c5341129348edf00a95dc87" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:04:12.468894 26591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce19ea795cc849b599b5e3a69365f711. Latest consensus state: current_term: 1 leader_uuid: "ce19ea795cc849b599b5e3a69365f711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce19ea795cc849b599b5e3a69365f711" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } } peers { permanent_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } } peers { permanent_uuid: "2b7bac880c5341129348edf00a95dc87" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } } }
I20260808 23:04:12.468972 26591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:04:12.470419 26612 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce19ea795cc849b599b5e3a69365f711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce19ea795cc849b599b5e3a69365f711" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } } peers { permanent_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } } peers { permanent_uuid: "2b7bac880c5341129348edf00a95dc87" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } } }
I20260808 23:04:12.470491 26612 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [sys.catalog]: This master's current role is: LEADER
I20260808 23:04:12.470878 26591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce19ea795cc849b599b5e3a69365f711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce19ea795cc849b599b5e3a69365f711" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } } peers { permanent_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } } peers { permanent_uuid: "2b7bac880c5341129348edf00a95dc87" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } } }
I20260808 23:04:12.470952 26591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:04:12.471227 26647 mvcc.cc:205] Tried to move back new op lower bound from 7316399114100002816 to 7316399114089054208. Current Snapshot: MvccSnapshot[applied={T|T < 7316399114100002816}]
I20260808 23:04:12.471688 26658 mvcc.cc:205] Tried to move back new op lower bound from 7316399114100002816 to 7316399114089054208. Current Snapshot: MvccSnapshot[applied={T|T < 7316399114100002816}]
I20260808 23:04:12.472234 26643 catalog_manager.cc:1347] Generated new cluster ID: 8e4581e529f14c01851cae2fa391f782
I20260808 23:04:12.472276 26643 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:04:12.475044 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.466801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43420 (local address 127.23.194.61:34183)
0808 23:04:12.467378 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.467380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.467389 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:04:12.467409 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.467410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.467420 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.467473 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.468149 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.468890 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.469931 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.470242 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.470280 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.471865 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:12.471866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:12.471867 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:12.471873 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:12.473886 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.474247 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.474248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.474249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.474305 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.474491 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.474492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.474492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.474604 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:12.474679 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.474885 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.474958 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541}
I20260808 23:04:12.475630 26587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.465978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42096 (local address 127.23.194.60:44673)
0808 23:04:12.466013 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.466015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.466109 (+ 94us) server_negotiation.cc:407] Connection header received
0808 23:04:12.466222 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.466222 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.466233 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.466292 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.467512 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.468151 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.469616 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.469887 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.469932 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.472515 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:12.472516 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:12.472517 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:12.472535 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:12.474097 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.474633 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.474635 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.474635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.474691 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.474959 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.474960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.474960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.475077 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:12.475147 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.475482 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.475556 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 23:04:12.479058 26472 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } attempt: 3
I20260808 23:04:12.482163 26358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.466548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41694 (local address 127.23.194.62:39523)
0808 23:04:12.466583 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.466585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.475362 (+ 8777us) server_negotiation.cc:407] Connection header received
0808 23:04:12.475449 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.475450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.475459 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.475485 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.476031 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.476705 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.477675 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.477971 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.478011 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.479381 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:12.479382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:12.479383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:12.479389 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:12.480913 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.481276 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.481277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.481278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.481332 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.481579 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.481580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.481580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.481711 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:12.481793 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.482005 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.482080 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 08 23:04:12 dist-test-slave-f0tn krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230252, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.61@KRBTEST.COM for kudu/127.23.194.60@KRBTEST.COM
I20260808 23:04:12.486284 26643 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:04:12.486708 26477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.479270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:44673 (local address 127.0.0.1:42098)
0808 23:04:12.479336 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:12.479344 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 23:04:12.479375 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:04:12.479577 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:04:12.479577 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:04:12.479938 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:04:12.479975 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:12.479977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.480807 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.480807 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:12.481586 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:12.481587 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.482080 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.482081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:12.482147 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.482621 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:04:12.482626 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:04:12.484008 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:04:12.485632 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:12.485633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:12.485633 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:12.485838 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:12.486017 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:12.486018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:12.486018 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:12.486091 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:12.486411 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:04:12.486412 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:04:12.486455 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 23:04:12.486512 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 23:04:12.486602 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 23:04:12.487154 26587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.479426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42098 (local address 127.23.194.60:44673)
0808 23:04:12.479469 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.479470 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.479475 (+ 5us) server_negotiation.cc:407] Connection header received
0808 23:04:12.479491 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.479491 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.479512 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.479546 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.480060 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.480721 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.481681 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.481980 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.482024 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.484099 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:12.484100 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:12.484101 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:12.484107 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:12.485546 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.485911 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.485912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.485912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.485963 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:12.486171 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:12.486172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:12.486172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:12.486294 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:12.486957 (+ 663us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.487000 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.487069 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 23:04:12.487350 26643 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:04:12.488811 26472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293: Bootstrap starting.
I20260808 23:04:12.489522 26472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293: Neither blocks nor log segments found. Creating new log.
I20260808 23:04:12.489848 26472 log.cc:824] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293: Log is configured to *not* fsync() on all Append() calls
I20260808 23:04:12.490545 26643 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711: Generated new TSK 0
I20260808 23:04:12.490702 26472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293: No bootstrap required, opened a new log
I20260808 23:04:12.490708 26643 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:04:12.492653 26472 raft_consensus.cc:348] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce19ea795cc849b599b5e3a69365f711" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } } peers { permanent_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } } peers { permanent_uuid: "2b7bac880c5341129348edf00a95dc87" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } }
I20260808 23:04:12.492785 26472 raft_consensus.cc:729] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c871319bdb7a44cba7c0d6d0f8bb8293, State: Initialized, Role: FOLLOWER
I20260808 23:04:12.492861 26472 consensus_queue.cc:260] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce19ea795cc849b599b5e3a69365f711" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } } peers { permanent_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } } peers { permanent_uuid: "2b7bac880c5341129348edf00a95dc87" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } }
I20260808 23:04:12.493165 26472 sys_catalog.cc:564] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:04:12.493505 26424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:04:12.494347 26424 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293 [term 1 FOLLOWER]: Refusing update from remote peer ce19ea795cc849b599b5e3a69365f711: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:04:12.494663 26612 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:04:12.494832 26666 sys_catalog.cc:455] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce19ea795cc849b599b5e3a69365f711" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } } peers { permanent_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } } peers { permanent_uuid: "2b7bac880c5341129348edf00a95dc87" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } } }
I20260808 23:04:12.494866 26427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce19ea795cc849b599b5e3a69365f711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293"
I20260808 23:04:12.497618 26666 sys_catalog.cc:458] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:04:12.497691 26666 sys_catalog.cc:455] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce19ea795cc849b599b5e3a69365f711. Latest consensus state: current_term: 1 leader_uuid: "ce19ea795cc849b599b5e3a69365f711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce19ea795cc849b599b5e3a69365f711" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } } peers { permanent_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } } peers { permanent_uuid: "2b7bac880c5341129348edf00a95dc87" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } } }
I20260808 23:04:12.497745 26666 sys_catalog.cc:458] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:04:12.497771 26666 sys_catalog.cc:455] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce19ea795cc849b599b5e3a69365f711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce19ea795cc849b599b5e3a69365f711" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } } peers { permanent_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } } peers { permanent_uuid: "2b7bac880c5341129348edf00a95dc87" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } } }
I20260808 23:04:12.497805 26666 sys_catalog.cc:458] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 23:04:12.498073 26398 tablet.cc:2369] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:04:12.498337 26427 raft_consensus.cc:2396] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ce19ea795cc849b599b5e3a69365f711 for term 1 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260808 23:04:12.499029 26689 mvcc.cc:205] Tried to move back new op lower bound from 7316399114197725184 to 7316399114089054208. Current Snapshot: MvccSnapshot[applied={T|T < 7316399114100002816}]
I20260808 23:04:12.500922 26684 catalog_manager.cc:1259] Loaded cluster ID: 8e4581e529f14c01851cae2fa391f782
I20260808 23:04:12.500969 26684 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293: loading cluster ID for follower catalog manager: success
I20260808 23:04:12.502177 26684 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293: acquiring CA information for follower catalog manager: success
I20260808 23:04:12.502502 26684 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c871319bdb7a44cba7c0d6d0f8bb8293: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 23:04:12.526283 26587 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:12.521276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42112 (local address 127.23.194.60:44673)
0808 23:04:12.521316 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.521319 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.525586 (+ 4267us) server_negotiation.cc:407] Connection header received
0808 23:04:12.525712 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.525713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.525724 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.525754 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.526218 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42112: BlockingRecv error: recv got EOF from 127.0.0.1:42112 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 23:04:12.527354 26483 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:12.525617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43434 (local address 127.23.194.61:34183)
0808 23:04:12.525657 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.525659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.526744 (+ 1085us) server_negotiation.cc:407] Connection header received
0808 23:04:12.526935 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.526936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.526947 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.526976 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.527294 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43434: BlockingRecv error: recv got EOF from 127.0.0.1:43434 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 23:04:12.528070 26358 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:12.526576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41708 (local address 127.23.194.62:39523)
0808 23:04:12.526616 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.526618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.527558 (+ 940us) server_negotiation.cc:407] Connection header received
0808 23:04:12.527669 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.527670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.527681 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.527710 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.528011 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41708: BlockingRecv error: recv got EOF from 127.0.0.1:41708 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 23:04:12.539760 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.536178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43438 (local address 127.23.194.61:34183)
0808 23:04:12.536226 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.536228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.536491 (+ 263us) server_negotiation.cc:407] Connection header received
0808 23:04:12.536628 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.536629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.536679 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.536723 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:12.536925 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.537708 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.538809 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.539086 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.539133 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.539371 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:12.539487 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:12.539539 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.539675 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.539694 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 23:04:12.542955 26587 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:12.542205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42120 (local address 127.23.194.60:44673)
0808 23:04:12.542240 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.542242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.542425 (+ 183us) server_negotiation.cc:407] Connection header received
0808 23:04:12.542526 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.542527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.542543 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.542572 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.542907 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42120: BlockingRecv error: recv got EOF from 127.0.0.1:42120 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 23:04:12.548795 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.545726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43440 (local address 127.23.194.61:34183)
0808 23:04:12.545758 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.545759 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.545974 (+ 215us) server_negotiation.cc:407] Connection header received
0808 23:04:12.546064 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.546065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.546073 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.546100 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:12.546273 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.546956 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.548015 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.548288 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.548331 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.548540 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:12.548572 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:12.548599 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.548716 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.548734 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 23:04:12.551867 26587 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:12.550979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42124 (local address 127.23.194.60:44673)
0808 23:04:12.551012 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.551014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.551377 (+ 363us) server_negotiation.cc:407] Connection header received
0808 23:04:12.551467 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.551469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.551478 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.551503 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.551823 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42124: BlockingRecv error: recv got EOF from 127.0.0.1:42124 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 23:04:12.557965 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.554871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43446 (local address 127.23.194.61:34183)
0808 23:04:12.554914 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.554915 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.555125 (+ 210us) server_negotiation.cc:407] Connection header received
0808 23:04:12.555215 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.555215 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.555224 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.555250 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:12.555402 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.556094 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.557166 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.557471 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.557513 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.557724 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:12.557757 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:12.557782 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.557883 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.557899 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260808 23:04:12.560645 26587 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:12.559909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42136 (local address 127.23.194.60:44673)
0808 23:04:12.559940 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.559942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.560167 (+ 225us) server_negotiation.cc:407] Connection header received
0808 23:04:12.560259 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.560260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.560270 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.560297 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.560590 (+ 293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42136: BlockingRecv error: recv got EOF from 127.0.0.1:42136 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 23:04:12.568586 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.565612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43458 (local address 127.23.194.61:34183)
0808 23:04:12.565653 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.565655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.565895 (+ 240us) server_negotiation.cc:407] Connection header received
0808 23:04:12.565987 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.565988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.565996 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.566024 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:12.566182 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.566861 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.567881 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.568149 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.568186 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.568361 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:12.568390 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:12.568414 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.568509 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.568524 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 23:04:12.571380 26587 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:12.570513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42148 (local address 127.23.194.60:44673)
0808 23:04:12.570545 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.570546 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.570905 (+ 359us) server_negotiation.cc:407] Connection header received
0808 23:04:12.571004 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.571005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.571015 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.571041 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.571335 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42148: BlockingRecv error: recv got EOF from 127.0.0.1:42148 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 23:04:12.583920 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.580415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43462 (local address 127.23.194.61:34183)
0808 23:04:12.580456 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.580457 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.580738 (+ 281us) server_negotiation.cc:407] Connection header received
0808 23:04:12.580835 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.580836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.580846 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.580873 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:12.581048 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.581816 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.583071 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.583405 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.583451 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.583665 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:12.583700 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:12.583726 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.583825 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.583846 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 23:04:12.587069 26587 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:12.586270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42158 (local address 127.23.194.60:44673)
0808 23:04:12.586307 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.586309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.586557 (+ 248us) server_negotiation.cc:407] Connection header received
0808 23:04:12.586654 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.586655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.586666 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.586694 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.587016 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42158: BlockingRecv error: recv got EOF from 127.0.0.1:42158 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 23:04:12.607602 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.604466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43470 (local address 127.23.194.61:34183)
0808 23:04:12.604507 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.604509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.604813 (+ 304us) server_negotiation.cc:407] Connection header received
0808 23:04:12.604907 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.604908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.604920 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.604944 (+ 24us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:12.605106 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.605771 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.606839 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.607111 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.607152 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.607368 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:12.607399 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:12.607444 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.607503 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.607522 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 23:04:12.610016 26587 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:12.609190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42172 (local address 127.23.194.60:44673)
0808 23:04:12.609226 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.609228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.609560 (+ 332us) server_negotiation.cc:407] Connection header received
0808 23:04:12.609618 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.609619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.609630 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.609675 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.609963 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42172: BlockingRecv error: recv got EOF from 127.0.0.1:42172 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 23:04:12.646540 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.643093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43484 (local address 127.23.194.61:34183)
0808 23:04:12.643154 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.643157 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.643467 (+ 310us) server_negotiation.cc:407] Connection header received
0808 23:04:12.643572 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.643574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.643585 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.643614 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:12.643802 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.644555 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.645734 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.646026 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.646066 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.646270 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:12.646307 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:12.646334 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.646434 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.646455 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 23:04:12.650938 26587 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:12.650184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42176 (local address 127.23.194.60:44673)
0808 23:04:12.650247 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.650249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.650256 (+ 7us) server_negotiation.cc:407] Connection header received
0808 23:04:12.650282 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.650283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.650294 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.650363 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.650848 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42176: BlockingRecv error: recv got EOF from 127.0.0.1:42176 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 23:04:12.718703 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.716423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43488 (local address 127.23.194.61:34183)
0808 23:04:12.716472 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.716475 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.716583 (+ 108us) server_negotiation.cc:407] Connection header received
0808 23:04:12.716652 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.716654 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.716665 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.716716 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:12.716838 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.717321 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.718082 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.718276 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.718333 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.718469 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:12.718501 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:12.718542 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.718601 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.718621 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 23:04:12.720721 26587 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:12.720256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42188 (local address 127.23.194.60:44673)
0808 23:04:12.720313 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.720315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.720328 (+ 13us) server_negotiation.cc:407] Connection header received
0808 23:04:12.720385 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.720386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.720394 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.720440 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.720678 (+ 238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42188: BlockingRecv error: recv got EOF from 127.0.0.1:42188 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 23:04:12.852097 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:12.849860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43490 (local address 127.23.194.61:34183)
0808 23:04:12.849936 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.849939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.850014 (+ 75us) server_negotiation.cc:407] Connection header received
0808 23:04:12.850090 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.850091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.850102 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.850148 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:12.850274 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.850755 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.851492 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:12.851680 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:12.851735 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:12.851863 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:12.851894 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:12.851934 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:12.851990 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 23:04:12.852012 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 23:04:12.854140 26587 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:12.853690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42196 (local address 127.23.194.60:44673)
0808 23:04:12.853732 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:12.853734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:12.853743 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:04:12.853786 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:12.853787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:12.853796 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:12.853838 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:12.854079 (+ 241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42196: BlockingRecv error: recv got EOF from 127.0.0.1:42196 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 23:04:13.113294 26612 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce19ea795cc849b599b5e3a69365f711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce19ea795cc849b599b5e3a69365f711" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 39523 } } peers { permanent_uuid: "c871319bdb7a44cba7c0d6d0f8bb8293" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 34183 } } peers { permanent_uuid: "2b7bac880c5341129348edf00a95dc87" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 44673 } } }
I20260808 23:04:13.113649 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:13.111270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43506 (local address 127.23.194.61:34183)
0808 23:04:13.111322 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:13.111324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:13.111400 (+ 76us) server_negotiation.cc:407] Connection header received
0808 23:04:13.111461 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:13.111462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:13.111470 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:13.111526 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:13.111672 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:13.112153 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:13.112873 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:13.113054 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:13.113102 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:13.113273 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:13.113308 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:13.113355 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:13.113557 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 23:04:13.113576 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 23:04:13.113682 26612 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce19ea795cc849b599b5e3a69365f711 [sys.catalog]: This master's current role is: LEADER
W20260808 23:04:13.115628 26587 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:13.115197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42208 (local address 127.23.194.60:44673)
0808 23:04:13.115244 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:13.115245 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:13.115254 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:04:13.115296 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:13.115296 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:13.115306 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:13.115346 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:13.115580 (+ 234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42208: BlockingRecv error: recv got EOF from 127.0.0.1:42208 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 23:04:13.432533 26607 catalog_manager.cc:1259] Loaded cluster ID: 8e4581e529f14c01851cae2fa391f782
I20260808 23:04:13.432610 26607 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87: loading cluster ID for follower catalog manager: success
I20260808 23:04:13.433452 26607 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87: acquiring CA information for follower catalog manager: success
I20260808 23:04:13.433780 26607 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2b7bac880c5341129348edf00a95dc87: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 23:04:13.630900 26872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:13.628680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43508 (local address 127.23.194.61:34183)
0808 23:04:13.628777 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:13.628779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:13.628808 (+ 29us) server_negotiation.cc:407] Connection header received
0808 23:04:13.628852 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:13.628853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:13.628864 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:13.628912 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:13.629012 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:13.629545 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:13.630328 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:13.630520 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:13.630570 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:13.630684 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:13.630715 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:13.630747 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:13.630791 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 23:04:13.630811 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":52,"threads_started":1}
I20260808 23:04:13.634532 26880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:13.632414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42210 (local address 127.23.194.60:44673)
0808 23:04:13.632508 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:13.632510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:13.632555 (+ 45us) server_negotiation.cc:407] Connection header received
0808 23:04:13.632606 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:13.632607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:13.632662 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:13.632725 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:13.632799 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:13.633262 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:13.633938 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:13.634126 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:13.634179 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:13.634276 (+ 97us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:13.634381 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:13.634431 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:13.634443 (+ 12us) server_negotiation.cc:299] Negotiation successful
0808 23:04:13.634458 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":53,"threads_started":1}
I20260808 23:04:13.638859 26872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:13.636082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43514 (local address 127.23.194.61:34183)
0808 23:04:13.636182 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:13.636184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:13.636293 (+ 109us) server_negotiation.cc:407] Connection header received
0808 23:04:13.636405 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:13.636406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:13.636418 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:13.636475 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:13.636624 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:13.637308 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:13.638120 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:13.638380 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:13.638450 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:13.638598 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:13.638631 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:13.638667 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:13.638760 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 23:04:13.638787 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 23:04:13.639199 26880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:13.636135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42212 (local address 127.23.194.60:44673)
0808 23:04:13.636169 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:13.636172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:13.636216 (+ 44us) server_negotiation.cc:407] Connection header received
0808 23:04:13.636291 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:13.636292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:13.636302 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:13.636345 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:13.636471 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:13.637189 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:13.638226 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:13.638503 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:13.638559 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:13.638715 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:13.638747 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:13.638792 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:13.639103 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 23:04:13.639130 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 23:04:13.639994 26892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:13.636012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41722 (local address 127.23.194.62:39523)
0808 23:04:13.636206 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:13.636208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:13.636360 (+ 152us) server_negotiation.cc:407] Connection header received
0808 23:04:13.636423 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:13.636424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:13.636532 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:13.636626 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:13.637541 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:13.638276 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:13.639142 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:13.639476 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:13.639531 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:13.639713 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:13.639792 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:13.639841 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:13.639889 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 23:04:13.639914 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":47,"threads_started":1}
I20260808 23:04:13.641480 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 26238
I20260808 23:04:13.654130 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 26366
I20260808 23:04:13.663316 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 26486
2026-08-08T23:04:13Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2920 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:04:13 dist-test-slave-f0tn krb5kdc[26897](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:04:13 dist-test-slave-f0tn krb5kdc[26897](info): set up 2 sockets
Aug 08 23:04:13 dist-test-slave-f0tn krb5kdc[26897](info): commencing operation
krb5kdc: starting...
W20260808 23:04:14.806646 24328 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/test-user.keytab.
Aug 08 23:04:14 dist-test-slave-f0tn krb5kdc[26897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:04:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:04:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
Entry for principal kudu/127.23.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab.
I20260808 23:04:14.918141 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:42245
--webserver_interface=127.23.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:38623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:42245,127.23.194.61:41227,127.23.194.60:42917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:04:14.997411 26917 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 23:04:14.997623 26917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:04:14.997646 26917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:04:14.997669 26917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:04:14.998788 26917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:04:14.998829 26917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:04:14.998847 26917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:04:14.998863 26917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:04:14.998878 26917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:04:15.000380 26917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:38623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:42245,127.23.194.61:41227,127.23.194.60:42917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.62.keytab
--principal=kudu/127.23.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.62:42245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:04:15.000658 26917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:04:15.000910 26917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:04:15.005723 26924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:15.005738 26928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:15.005736 26922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:04:15.005959 26917 server_base.cc:1034] running on GCE node
I20260808 23:04:15.006175 26917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:04:15.006453 26917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:04:15.007582 26917 hybrid_clock.cc:648] HybridClock initialized: now 1786230255007574 us; error 29 us; skew 500 ppm
Aug 08 23:04:15 dist-test-slave-f0tn krb5kdc[26897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230255, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:04:15.013021 26917 init.cc:377] Logged in from keytab as kudu/127.23.194.62@KRBTEST.COM (short username kudu)
I20260808 23:04:15.013460 26917 webserver.cc:460] Webserver started at http://127.23.194.62:43561/ using document root <none> and password file <none>
I20260808 23:04:15.013645 26917 fs_manager.cc:359] Metadata directory not provided
I20260808 23:04:15.013684 26917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:04:15.013775 26917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:04:15.014776 26917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-0/data/instance:
uuid: "3d02a10f30e743a5bdb328bc6072fc93"
format_stamp: "Formatted at 2026-08-08 23:04:15 on dist-test-slave-f0tn"
server_key: "f06a63214f8355b2d3d89b8365248d97"
server_key_iv: "09f101e726cb37a3c290a9143d525d3f"
server_key_version: "encryptionkey@0"
I20260808 23:04:15.015064 26917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-0/wal/instance:
uuid: "3d02a10f30e743a5bdb328bc6072fc93"
format_stamp: "Formatted at 2026-08-08 23:04:15 on dist-test-slave-f0tn"
server_key: "f06a63214f8355b2d3d89b8365248d97"
server_key_iv: "09f101e726cb37a3c290a9143d525d3f"
server_key_version: "encryptionkey@0"
I20260808 23:04:15.016263 26917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 23:04:15.016940 26946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:04:15.017102 26917 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 23:04:15.017190 26917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-0/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-0/wal
uuid: "3d02a10f30e743a5bdb328bc6072fc93"
format_stamp: "Formatted at 2026-08-08 23:04:15 on dist-test-slave-f0tn"
server_key: "f06a63214f8355b2d3d89b8365248d97"
server_key_iv: "09f101e726cb37a3c290a9143d525d3f"
server_key_version: "encryptionkey@0"
I20260808 23:04:15.017246 26917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:04:15.031996 26917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:04:15.033746 26917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:04:15.033855 26917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:04:15.037784 26917 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.62:42245
I20260808 23:04:15.037819 27021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.62:42245 every 8 connection(s)
I20260808 23:04:15.038144 26917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-0/data/info.pb
I20260808 23:04:15.039122 27024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:04:15.040619 27024 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:04:15.042280 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 26917
I20260808 23:04:15.042369 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-0/wal/instance
I20260808 23:04:15.042524 24328 external_mini_cluster.cc:1428] Setting key da40490b65a97f98f9f2b1a94f0ea7bd
Aug 08 23:04:15 dist-test-slave-f0tn krb5kdc[26897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230255, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:04:15.050379 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.041236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:42245 (local address 127.0.0.1:47816)
0808 23:04:15.041459 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.041497 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:04:15.041964 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:04:15.042208 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:04:15.042210 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:04:15.042485 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:04:15.042675 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:15.042680 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.043986 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.043987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:15.044641 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:15.044643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.045295 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.045296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:15.045354 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.045870 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:04:15.045879 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:04:15.047199 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:04:15.049034 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:15.049040 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:15.049042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:15.049227 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:15.049504 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:15.049505 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:15.049505 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:15.049579 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:15.049979 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:04:15.049982 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:04:15.050030 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 23:04:15.050091 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 23:04:15.050221 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":108,"threads_started":1}
I20260808 23:04:15.050400 27027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.041283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47816 (local address 127.23.194.62:42245)
0808 23:04:15.041473 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.041482 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.041579 (+ 97us) server_negotiation.cc:407] Connection header received
0808 23:04:15.042076 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.042077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.042108 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.042209 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.042770 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.043905 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.044717 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.045186 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.045299 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.047283 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:15.047292 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:15.047292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:15.047300 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:15.048956 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.049312 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.049316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.049316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.049442 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.049638 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.049639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.049639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.049914 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:15.049997 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.050148 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.050299 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":159,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for kudu/127.23.194.61@KRBTEST.COM; defaulting to no policy
I20260808 23:04:15.051182 27024 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:04:15.051501 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.051378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:41227 (local address 127.0.0.1:50220)
0808 23:04:15.051428 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.051480 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.61:41227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 23:04:15.051630 26948 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.194.61:41227: connect: Connection refused (error 111)
I20260808 23:04:15.051998 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.051923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:41227 (local address 127.0.0.1:50232)
0808 23:04:15.051958 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.051983 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.61:41227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 23:04:15.052088 27024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.61:41227: Network error: Client connection negotiation failed: client connection to 127.23.194.61:41227: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.61.keytab.
Entry for principal kudu/127.23.194.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.61.keytab.
I20260808 23:04:15.065557 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.61:41227
--webserver_interface=127.23.194.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.61.keytab
--principal=kudu/127.23.194.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:38623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:42245,127.23.194.61:41227,127.23.194.60:42917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:04:15.101315 27024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } attempt: 1
I20260808 23:04:15.101871 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.101719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:41227 (local address 127.0.0.1:50234)
0808 23:04:15.101793 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.101837 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.61:41227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 23:04:15.102186 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.102114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:41227 (local address 127.0.0.1:50246)
0808 23:04:15.102146 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.102170 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.61:41227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260808 23:04:15.102268 27024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.61:41227: Network error: Client connection negotiation failed: client connection to 127.23.194.61:41227: connect: Connection refused (error 111)
W20260808 23:04:15.145228 27034 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 23:04:15.145435 27034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:04:15.145452 27034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:04:15.145473 27034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:04:15.146582 27034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:04:15.146622 27034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:04:15.146636 27034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:04:15.146660 27034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:04:15.146682 27034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:04:15.148180 27034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:38623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:42245,127.23.194.61:41227,127.23.194.60:42917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.61.keytab
--principal=kudu/127.23.194.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.61:41227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.194.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:04:15.148396 27034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:04:15.148597 27034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:04:15.153141 27045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:15.153179 27048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:15.153177 27044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:04:15.153306 27034 server_base.cc:1034] running on GCE node
I20260808 23:04:15.153625 27034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:04:15.153893 27034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:04:15.155030 27034 hybrid_clock.cc:648] HybridClock initialized: now 1786230255155014 us; error 32 us; skew 500 ppm
Aug 08 23:04:15 dist-test-slave-f0tn krb5kdc[26897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230255, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:04:15.157308 27034 init.cc:377] Logged in from keytab as kudu/127.23.194.61@KRBTEST.COM (short username kudu)
I20260808 23:04:15.157742 27034 webserver.cc:460] Webserver started at http://127.23.194.61:36901/ using document root <none> and password file <none>
I20260808 23:04:15.157953 27034 fs_manager.cc:359] Metadata directory not provided
I20260808 23:04:15.158000 27034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:04:15.158123 27034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:04:15.159152 27034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-1/data/instance:
uuid: "f198d1f307a24402a30ba695b9d7142e"
format_stamp: "Formatted at 2026-08-08 23:04:15 on dist-test-slave-f0tn"
server_key: "895d2dc88f9ceefb7fadb8907481ac56"
server_key_iv: "c4546c6c0e05483c5606586ccdde2d73"
server_key_version: "encryptionkey@0"
I20260808 23:04:15.159466 27034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-1/wal/instance:
uuid: "f198d1f307a24402a30ba695b9d7142e"
format_stamp: "Formatted at 2026-08-08 23:04:15 on dist-test-slave-f0tn"
server_key: "895d2dc88f9ceefb7fadb8907481ac56"
server_key_iv: "c4546c6c0e05483c5606586ccdde2d73"
server_key_version: "encryptionkey@0"
I20260808 23:04:15.160638 27034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:04:15.161266 27055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:04:15.161433 27034 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 23:04:15.161516 27034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-1/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-1/wal
uuid: "f198d1f307a24402a30ba695b9d7142e"
format_stamp: "Formatted at 2026-08-08 23:04:15 on dist-test-slave-f0tn"
server_key: "895d2dc88f9ceefb7fadb8907481ac56"
server_key_iv: "c4546c6c0e05483c5606586ccdde2d73"
server_key_version: "encryptionkey@0"
I20260808 23:04:15.161579 27034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:04:15.170432 27024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } attempt: 2
I20260808 23:04:15.170797 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.170689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:41227 (local address 127.0.0.1:50252)
0808 23:04:15.170741 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.170768 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.61:41227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 23:04:15.171126 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.171046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:41227 (local address 127.0.0.1:50258)
0808 23:04:15.171084 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.171108 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.61:41227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 23:04:15.171204 27024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.61:41227: Network error: Client connection negotiation failed: client connection to 127.23.194.61:41227: connect: Connection refused (error 111)
I20260808 23:04:15.176512 27034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:04:15.178092 27034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:04:15.178197 27034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:04:15.181891 27034 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.61:41227
I20260808 23:04:15.181929 27131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.61:41227 every 8 connection(s)
I20260808 23:04:15.182229 27034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-1/data/info.pb
I20260808 23:04:15.183135 27133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:04:15.184609 27133 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:04:15 dist-test-slave-f0tn krb5kdc[26897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230255, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.61@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:04:15.189815 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 27034
I20260808 23:04:15.189899 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-1/wal/instance
I20260808 23:04:15.190112 24328 external_mini_cluster.cc:1428] Setting key a37707e2a5b6c4d1558792ba5eab867c
I20260808 23:04:15.192601 27027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.185234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47828 (local address 127.23.194.62:42245)
0808 23:04:15.185293 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.185295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.185465 (+ 170us) server_negotiation.cc:407] Connection header received
0808 23:04:15.185905 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.185906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.185915 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.185966 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.186498 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.186967 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.187846 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.188023 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.188064 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.189921 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:15.189923 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:15.189924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:15.189932 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:15.191335 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.191728 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.191730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.191730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.191797 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.192051 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.192052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.192052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.192151 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:15.192204 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.192481 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.192541 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 23:04:15.192812 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.185183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:42245 (local address 127.0.0.1:47828)
0808 23:04:15.185333 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.185372 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 23:04:15.185832 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:04:15.186017 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:04:15.186020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:04:15.186253 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:04:15.186437 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:15.186441 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.187025 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.187025 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:15.187789 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:15.187790 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.188082 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.188083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:15.188188 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.188559 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:04:15.188566 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:04:15.189827 (+ 1261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:04:15.191409 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:15.191417 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:15.191419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:15.191660 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:15.191859 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:15.191860 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:15.191860 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:15.191992 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:15.192210 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:04:15.192214 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:04:15.192363 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 23:04:15.192461 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 23:04:15.192606 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
I20260808 23:04:15.193343 27133 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:04:15 dist-test-slave-f0tn krb5kdc[26897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230255, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.61@KRBTEST.COM for kudu/127.23.194.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.194.60@KRBTEST.COM; defaulting to no policy
I20260808 23:04:15.200105 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.193589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:41227 (local address 127.0.0.1:50262)
0808 23:04:15.193633 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.193644 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 23:04:15.193684 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:04:15.193898 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:04:15.193899 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:04:15.194025 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:04:15.194078 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:15.194079 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.194941 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.194942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:15.195691 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:15.195694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.196099 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.196100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:15.196178 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.196484 (+ 306us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:04:15.196489 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:04:15.197645 (+ 1156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:04:15.199191 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:15.199192 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:15.199193 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:15.199340 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:15.199519 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:15.199520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:15.199520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:15.199569 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:15.199806 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:04:15.199807 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:04:15.199843 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 23:04:15.199904 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 23:04:15.199998 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 23:04:15.200189 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.193647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50262 (local address 127.23.194.61:41227)
0808 23:04:15.193776 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.193780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.193798 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:04:15.193817 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.193819 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.193828 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.193886 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.194140 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.194862 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.195762 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.196037 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.196081 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.197716 (+ 1635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:15.197724 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:15.197725 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:15.197731 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:15.199115 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.199416 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.199419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.199419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.199467 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.199617 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.199618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.199618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.199748 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:15.199817 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.199946 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.200066 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":102,"thread_start_us":68,"threads_started":1}
I20260808 23:04:15.200706 27133 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.194.60@KRBTEST.COM" created.
I20260808 23:04:15.201018 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.200907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:42917 (local address 127.0.0.1:47530)
0808 23:04:15.200948 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.200996 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 23:04:15.202354 27061 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.194.60:42917: connect: Connection refused (error 111)
I20260808 23:04:15.202749 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.202648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:42917 (local address 127.0.0.1:47536)
0808 23:04:15.202694 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.202725 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 23:04:15.202880 27133 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.60:42917: Network error: Client connection negotiation failed: client connection to 127.23.194.60:42917: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.194.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.60.keytab.
Entry for principal kudu/127.23.194.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.60.keytab.
I20260808 23:04:15.214318 24328 external_mini_cluster.cc:1326] Running /tmp/dist-test-taski6UDxc/build/release/bin/kudu
/tmp/dist-test-taski6UDxc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.60:42917
--webserver_interface=127.23.194.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.60.keytab
--principal=kudu/127.23.194.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.194.20:38623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.194.62:42245,127.23.194.61:41227,127.23.194.60:42917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:04:15.252102 27133 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } attempt: 1
I20260808 23:04:15.252566 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.252442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:42917 (local address 127.0.0.1:47542)
0808 23:04:15.252501 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.252535 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 23:04:15.252871 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.252792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:42917 (local address 127.0.0.1:47552)
0808 23:04:15.252827 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.252850 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 23:04:15.252956 27133 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.60:42917: Network error: Client connection negotiation failed: client connection to 127.23.194.60:42917: connect: Connection refused (error 111)
I20260808 23:04:15.262355 27024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } attempt: 3
Aug 08 23:04:15 dist-test-slave-f0tn krb5kdc[26897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230255, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for kudu/127.23.194.61@KRBTEST.COM
I20260808 23:04:15.269366 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.262750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50266 (local address 127.23.194.61:41227)
0808 23:04:15.262808 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.262810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.262818 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:04:15.262843 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.262843 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.262851 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.262895 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.263201 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.263711 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.264794 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.265064 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.265130 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.267091 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:15.267092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:15.267092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:15.267099 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:15.268397 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.268753 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.268754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.268754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.268803 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.268960 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.268961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.268961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.269057 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:15.269098 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.269241 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.269300 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 23:04:15.269445 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.262664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:41227 (local address 127.0.0.1:50266)
0808 23:04:15.262713 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.262723 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:04:15.262763 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:04:15.262907 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:04:15.262908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:04:15.263063 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:04:15.263127 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:15.263129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.263850 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.263852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:15.264707 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:15.264710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.265141 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.265142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:15.265228 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.265665 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:04:15.265671 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:04:15.267011 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:04:15.268472 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:15.268474 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:15.268474 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:15.268691 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:15.268852 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:15.268852 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:15.268852 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:15.268911 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:15.269106 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:04:15.269108 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:04:15.269156 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 23:04:15.269226 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 23:04:15.269321 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 23:04:15.269891 27024 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:04:15.270223 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.270119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:42917 (local address 127.0.0.1:47554)
0808 23:04:15.270165 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.270200 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 23:04:15.270551 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.270468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:42917 (local address 127.0.0.1:47568)
0808 23:04:15.270504 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.270531 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 23:04:15.270639 27024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.60:42917: Network error: Client connection negotiation failed: client connection to 127.23.194.60:42917: connect: Connection refused (error 111)
W20260808 23:04:15.298473 27149 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 23:04:15.298754 27149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:04:15.298784 27149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:04:15.298822 27149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:04:15.300408 27149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:04:15.300464 27149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:04:15.300483 27149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:04:15.300499 27149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:04:15.300571 27149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:04:15.301772 27024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } attempt: 1
I20260808 23:04:15.302172 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.302049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:42917 (local address 127.0.0.1:47584)
0808 23:04:15.302098 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.302139 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 23:04:15.302443 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.302378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:42917 (local address 127.0.0.1:47588)
0808 23:04:15.302403 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.302425 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260808 23:04:15.302508 27024 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.60:42917: Network error: Client connection negotiation failed: client connection to 127.23.194.60:42917: connect: Connection refused (error 111)
I20260808 23:04:15.302700 27149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.194.20:38623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.194.62:42245,127.23.194.61:41227,127.23.194.60:42917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/krb5kdc/kudu_127.23.194.60.keytab
--principal=kudu/127.23.194.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.194.60:42917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.194.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 21:59:43 UTC on ccycloud.quasar-plhyte.root.comops.site
build id 4575490
I20260808 23:04:15.302947 27149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:04:15.303194 27149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:04:15.308238 27156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:15.308233 27157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:04:15.308329 27159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:04:15.308846 27149 server_base.cc:1034] running on GCE node
I20260808 23:04:15.309104 27149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:04:15.309459 27149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:04:15.310628 27149 hybrid_clock.cc:648] HybridClock initialized: now 1786230255310610 us; error 33 us; skew 500 ppm
Aug 08 23:04:15 dist-test-slave-f0tn krb5kdc[26897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230255, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:04:15.313067 27149 init.cc:377] Logged in from keytab as kudu/127.23.194.60@KRBTEST.COM (short username kudu)
I20260808 23:04:15.313555 27149 webserver.cc:460] Webserver started at http://127.23.194.60:43311/ using document root <none> and password file <none>
I20260808 23:04:15.313833 27149 fs_manager.cc:359] Metadata directory not provided
I20260808 23:04:15.313887 27149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:04:15.314020 27149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:04:15.315368 27149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-2/data/instance:
uuid: "d33dbeb2564646bb8fc9d5d262f643c7"
format_stamp: "Formatted at 2026-08-08 23:04:15 on dist-test-slave-f0tn"
server_key: "ecdd72a256c5908498f01e95b17d06f3"
server_key_iv: "d5fa5cea12d57efc6836343061e6e4f0"
server_key_version: "encryptionkey@0"
I20260808 23:04:15.315752 27149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-2/wal/instance:
uuid: "d33dbeb2564646bb8fc9d5d262f643c7"
format_stamp: "Formatted at 2026-08-08 23:04:15 on dist-test-slave-f0tn"
server_key: "ecdd72a256c5908498f01e95b17d06f3"
server_key_iv: "d5fa5cea12d57efc6836343061e6e4f0"
server_key_version: "encryptionkey@0"
I20260808 23:04:15.317162 27149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 23:04:15.318056 27168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:04:15.318213 27149 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 23:04:15.318287 27149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-2/data,/tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-2/wal
uuid: "d33dbeb2564646bb8fc9d5d262f643c7"
format_stamp: "Formatted at 2026-08-08 23:04:15 on dist-test-slave-f0tn"
server_key: "ecdd72a256c5908498f01e95b17d06f3"
server_key_iv: "d5fa5cea12d57efc6836343061e6e4f0"
server_key_version: "encryptionkey@0"
I20260808 23:04:15.318357 27149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:04:15.321094 27133 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } attempt: 2
I20260808 23:04:15.321525 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.321404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:42917 (local address 127.0.0.1:47592)
0808 23:04:15.321458 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.321492 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 23:04:15.321839 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.321756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:42917 (local address 127.0.0.1:47600)
0808 23:04:15.321790 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.321813 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.194.60:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260808 23:04:15.321925 27133 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.194.60:42917: Network error: Client connection negotiation failed: client connection to 127.23.194.60:42917: connect: Connection refused (error 111)
I20260808 23:04:15.331214 27149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:04:15.333137 27149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:04:15.333271 27149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:04:15.337061 27149 rpc_server.cc:307] RPC server started. Bound to: 127.23.194.60:42917
I20260808 23:04:15.337100 27255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.194.60:42917 every 8 connection(s)
I20260808 23:04:15.337510 27149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-2/data/info.pb
I20260808 23:04:15.338493 27256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:04:15.339213 24328 external_mini_cluster.cc:1388] Started /tmp/dist-test-taski6UDxc/build/release/bin/kudu as pid 27149
I20260808 23:04:15.339313 24328 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taski6UDxc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786229981209642-24328-0/minicluster-data/master-2/wal/instance
I20260808 23:04:15.339514 24328 external_mini_cluster.cc:1428] Setting key c6f758887cefbaaeb2da34bf9b572cd9
I20260808 23:04:15.340298 27256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:04:15 dist-test-slave-f0tn krb5kdc[26897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230255, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.60@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
Aug 08 23:04:15 dist-test-slave-f0tn krb5kdc[26897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.60@KRBTEST.COM
I20260808 23:04:15.348690 27027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.341292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47834 (local address 127.23.194.62:42245)
0808 23:04:15.341345 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.341347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.341527 (+ 180us) server_negotiation.cc:407] Connection header received
0808 23:04:15.341588 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.341589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.341598 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.341638 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.342189 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.342622 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.343441 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.343618 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.343673 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.345877 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:15.345878 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:15.345879 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:15.345886 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:15.347326 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.347715 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.347716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.347717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.347773 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.348016 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.348017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.348017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.348136 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:15.348190 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.348492 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.348577 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 23:04:15.348965 27264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.341223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.62:42245 (local address 127.0.0.1:47834)
0808 23:04:15.341361 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.341469 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0808 23:04:15.341533 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:04:15.341663 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:04:15.341665 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:04:15.342001 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:04:15.342114 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:15.342120 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.342696 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.342697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:15.343364 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:15.343365 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.343701 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.343702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:15.343798 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.344294 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:04:15.344302 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:04:15.345793 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:04:15.347399 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:15.347406 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:15.347408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:15.347650 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:15.347825 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:15.347826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:15.347826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:15.347927 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:15.348195 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:04:15.348202 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:04:15.348348 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 23:04:15.348461 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 23:04:15.348595 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260808 23:04:15.349215 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.339990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47614 (local address 127.23.194.60:42917)
0808 23:04:15.340178 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.340188 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.340224 (+ 36us) server_negotiation.cc:407] Connection header received
0808 23:04:15.340656 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.340665 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.340728 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.340922 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.341497 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.342626 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.343346 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.343704 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.343798 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.346375 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:15.346382 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:15.346383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:15.346389 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:15.347911 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.348249 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.348252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.348252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.348301 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.348474 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.348475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.348475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.348632 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:15.348886 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.348926 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.348986 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260808 23:04:15.349537 27256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:04:15 dist-test-slave-f0tn krb5kdc[26897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230255, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.60@KRBTEST.COM for kudu/127.23.194.61@KRBTEST.COM
Aug 08 23:04:15 dist-test-slave-f0tn krb5kdc[26897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.62@KRBTEST.COM
I20260808 23:04:15.357143 27264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.349773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.61:41227 (local address 127.0.0.1:50276)
0808 23:04:15.350057 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.350073 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:04:15.350113 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:04:15.350248 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:04:15.350249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:04:15.350357 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:04:15.350401 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:15.350402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.351255 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.351256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:15.352002 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:15.352004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.352553 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.352554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:15.352628 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.353053 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:04:15.353059 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:04:15.354289 (+ 1230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:04:15.356000 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:15.356001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:15.356001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:15.356241 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:15.356518 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:15.356519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:15.356519 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:15.356590 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:15.356869 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:04:15.356870 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:04:15.356918 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 23:04:15.356967 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 23:04:15.357059 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269}
I20260808 23:04:15.357354 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.349839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50276 (local address 127.23.194.61:41227)
0808 23:04:15.349891 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.349893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.350101 (+ 208us) server_negotiation.cc:407] Connection header received
0808 23:04:15.350169 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.350170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.350182 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.350242 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.350452 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.351167 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.352112 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.352455 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.352537 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.354387 (+ 1850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:15.354390 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:15.354390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:15.354398 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:15.355916 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.356344 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.356346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.356347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.356428 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.356651 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.356651 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.356652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.356792 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:15.356870 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.357027 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.357129 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 23:04:15.357868 27256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:04:15.358385 27027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.350365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47846 (local address 127.23.194.62:42245)
0808 23:04:15.350397 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.350399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.350404 (+ 5us) server_negotiation.cc:407] Connection header received
0808 23:04:15.350421 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.350422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.350430 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.350471 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.351072 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.351740 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.352672 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.352913 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.352982 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.355278 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:15.355279 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:15.355280 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:15.355288 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:15.356748 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.357287 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.357290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.357290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.357370 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.357686 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.357687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.357688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.357848 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:15.358043 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.358189 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.358287 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 08 23:04:15 dist-test-slave-f0tn krb5kdc[26897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230255, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.60@KRBTEST.COM for kudu/127.23.194.60@KRBTEST.COM
Aug 08 23:04:15 dist-test-slave-f0tn krb5kdc[26897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.194.61@KRBTEST.COM
I20260808 23:04:15.365098 27264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.358270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:42917 (local address 127.0.0.1:47618)
0808 23:04:15.358327 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.358337 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 23:04:15.358374 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:04:15.358539 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:04:15.358541 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:04:15.358708 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:04:15.358754 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:15.358756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.359712 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.359714 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:15.360440 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:15.360443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.360849 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.360850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:15.360910 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.361404 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:04:15.361409 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:04:15.362609 (+ 1200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:04:15.364107 (+ 1498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:15.364108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:15.364108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:15.364272 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:15.364452 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:15.364452 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:15.364452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:15.364518 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:15.364770 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:04:15.364771 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:04:15.364812 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 23:04:15.364884 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 23:04:15.364986 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 23:04:15.365283 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.358335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47618 (local address 127.23.194.60:42917)
0808 23:04:15.358386 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.358388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.358396 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:04:15.358419 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.358420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.358432 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.358492 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.358843 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.359616 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.360582 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.360784 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.360836 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.362733 (+ 1897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:15.362734 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:15.362735 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:15.362743 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:15.364004 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.364337 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.364337 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.364338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.364390 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.364574 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.364575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.364575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.364705 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:15.364781 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.364911 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.365047 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":27}
I20260808 23:04:15.366751 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.359212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50292 (local address 127.23.194.61:41227)
0808 23:04:15.359273 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.359275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.359284 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:04:15.359307 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.359308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.359316 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.359365 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.359933 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.360421 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.361240 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.361547 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.361603 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.364064 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:15.364066 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:15.364066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:15.364074 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:15.365691 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.366038 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.366039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.366039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.366098 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.366256 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.366257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.366257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.366375 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:15.366550 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.366590 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.366662 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 23:04:15.367046 27256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7: Bootstrap starting.
I20260808 23:04:15.368121 27256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7: Neither blocks nor log segments found. Creating new log.
I20260808 23:04:15.368330 24328 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:04:15.368460 27256 log.cc:824] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7: Log is configured to *not* fsync() on all Append() calls
I20260808 23:04:15.369360 27256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7: No bootstrap required, opened a new log
I20260808 23:04:15.369585 26988 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3d02a10f30e743a5bdb328bc6072fc93"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47846
I20260808 23:04:15.370937 27256 raft_consensus.cc:348] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a10f30e743a5bdb328bc6072fc93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } } peers { permanent_uuid: "f198d1f307a24402a30ba695b9d7142e" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } } peers { permanent_uuid: "d33dbeb2564646bb8fc9d5d262f643c7" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } }
I20260808 23:04:15.371073 27256 raft_consensus.cc:729] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d33dbeb2564646bb8fc9d5d262f643c7, State: Initialized, Role: FOLLOWER
I20260808 23:04:15.371145 27256 consensus_queue.cc:260] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a10f30e743a5bdb328bc6072fc93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } } peers { permanent_uuid: "f198d1f307a24402a30ba695b9d7142e" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } } peers { permanent_uuid: "d33dbeb2564646bb8fc9d5d262f643c7" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } }
I20260808 23:04:15.371474 27256 sys_catalog.cc:564] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:04:15.371455 27266 sys_catalog.cc:455] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a10f30e743a5bdb328bc6072fc93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } } peers { permanent_uuid: "f198d1f307a24402a30ba695b9d7142e" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } } peers { permanent_uuid: "d33dbeb2564646bb8fc9d5d262f643c7" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } } }
I20260808 23:04:15.371564 27266 sys_catalog.cc:458] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 23:04:15.373273 27286 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:04:15.373340 27286 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:04:15.373687 27149 master_runner.cc:186] Error getting master registration for 127.23.194.62:42245: OK, instance_id { permanent_uuid: "3d02a10f30e743a5bdb328bc6072fc93" instance_seqno: 1786230255036636 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 23:04:15.376659 27149 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 23:04:15.377609 27024 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } attempt: 2
Aug 08 23:04:15 dist-test-slave-f0tn krb5kdc[26897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230255, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.62@KRBTEST.COM for kudu/127.23.194.60@KRBTEST.COM
I20260808 23:04:15.385986 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.378111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47622 (local address 127.23.194.60:42917)
0808 23:04:15.378163 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.378166 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.378174 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:04:15.378194 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.378195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.378208 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.378266 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.378627 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.379376 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.380375 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.380675 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.380732 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.383017 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:15.383018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:15.383019 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:15.383028 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:15.384673 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.385100 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.385102 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.385102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.385166 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.385344 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.385345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.385345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.385570 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:15.385634 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.385794 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.385881 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 23:04:15.385986 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.377880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:42917 (local address 127.0.0.1:47622)
0808 23:04:15.377951 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.377965 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:04:15.378028 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:04:15.378290 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:04:15.378291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:04:15.378491 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:04:15.378556 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:15.378559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.379458 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.379459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:15.380293 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:15.380296 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.380744 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.380746 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:15.380811 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.381328 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:04:15.381334 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:04:15.382916 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:04:15.384774 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:15.384775 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:15.384776 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:15.385019 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:15.385225 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:15.385227 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:15.385227 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:15.385289 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:15.385639 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:04:15.385640 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:04:15.385697 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 23:04:15.385762 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 23:04:15.385881 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 23:04:15.388139 27024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93: Bootstrap starting.
I20260808 23:04:15.388854 27024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93: Neither blocks nor log segments found. Creating new log.
I20260808 23:04:15.389151 27024 log.cc:824] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93: Log is configured to *not* fsync() on all Append() calls
I20260808 23:04:15.389938 27024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93: No bootstrap required, opened a new log
I20260808 23:04:15.391014 27024 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a10f30e743a5bdb328bc6072fc93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } } peers { permanent_uuid: "f198d1f307a24402a30ba695b9d7142e" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } } peers { permanent_uuid: "d33dbeb2564646bb8fc9d5d262f643c7" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } }
I20260808 23:04:15.391191 27024 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d02a10f30e743a5bdb328bc6072fc93, State: Initialized, Role: FOLLOWER
I20260808 23:04:15.391306 27024 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a10f30e743a5bdb328bc6072fc93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } } peers { permanent_uuid: "f198d1f307a24402a30ba695b9d7142e" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } } peers { permanent_uuid: "d33dbeb2564646bb8fc9d5d262f643c7" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } }
I20260808 23:04:15.391575 27290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a10f30e743a5bdb328bc6072fc93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } } peers { permanent_uuid: "f198d1f307a24402a30ba695b9d7142e" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } } peers { permanent_uuid: "d33dbeb2564646bb8fc9d5d262f643c7" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } } }
I20260808 23:04:15.391660 27024 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:04:15.391664 27290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:04:15.391736 26988 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 23:04:15.391816 26988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:04:15.392633 26988 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a10f30e743a5bdb328bc6072fc93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } } peers { permanent_uuid: "f198d1f307a24402a30ba695b9d7142e" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } } peers { permanent_uuid: "d33dbeb2564646bb8fc9d5d262f643c7" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } }
I20260808 23:04:15.393345 26988 leader_election.cc:288] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [CANDIDATE]: Term 1 election: Requested vote from peers f198d1f307a24402a30ba695b9d7142e (127.23.194.61:41227), d33dbeb2564646bb8fc9d5d262f643c7 (127.23.194.60:42917)
I20260808 23:04:15.393564 27212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d02a10f30e743a5bdb328bc6072fc93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d33dbeb2564646bb8fc9d5d262f643c7"
I20260808 23:04:15.393708 27212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:04:15.394644 27212 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d02a10f30e743a5bdb328bc6072fc93 in term 1.
I20260808 23:04:15.394922 26949 leader_election.cc:302] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d02a10f30e743a5bdb328bc6072fc93, d33dbeb2564646bb8fc9d5d262f643c7; no voters:
I20260808 23:04:15.395049 27290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:04:15.395174 27290 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [term 1 LEADER]: Becoming Leader. State: Replica: 3d02a10f30e743a5bdb328bc6072fc93, State: Running, Role: LEADER
I20260808 23:04:15.395253 27290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a10f30e743a5bdb328bc6072fc93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } } peers { permanent_uuid: "f198d1f307a24402a30ba695b9d7142e" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } } peers { permanent_uuid: "d33dbeb2564646bb8fc9d5d262f643c7" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } }
I20260808 23:04:15.395668 27290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d02a10f30e743a5bdb328bc6072fc93. Latest consensus state: current_term: 1 leader_uuid: "3d02a10f30e743a5bdb328bc6072fc93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a10f30e743a5bdb328bc6072fc93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } } peers { permanent_uuid: "f198d1f307a24402a30ba695b9d7142e" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } } peers { permanent_uuid: "d33dbeb2564646bb8fc9d5d262f643c7" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } } }
I20260808 23:04:15.395749 27290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [sys.catalog]: This master's current role is: LEADER
W20260808 23:04:15.396018 27308 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:04:15.396076 27308 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:04:15.396180 27321 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:04:15.396380 27321 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:04:15.394200 26917 master_runner.cc:186] Error getting master registration for 127.23.194.61:41227: OK, instance_id { permanent_uuid: "f198d1f307a24402a30ba695b9d7142e" instance_seqno: 1786230255180777 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 23:04:15.396498 26917 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 23:04:15.398381 27212 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7 [term 1 FOLLOWER]: Refusing update from remote peer 3d02a10f30e743a5bdb328bc6072fc93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:04:15.398765 27310 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d33dbeb2564646bb8fc9d5d262f643c7" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:04:15.400112 27266 sys_catalog.cc:455] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d02a10f30e743a5bdb328bc6072fc93. Latest consensus state: current_term: 1 leader_uuid: "3d02a10f30e743a5bdb328bc6072fc93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a10f30e743a5bdb328bc6072fc93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } } peers { permanent_uuid: "f198d1f307a24402a30ba695b9d7142e" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } } peers { permanent_uuid: "d33dbeb2564646bb8fc9d5d262f643c7" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } } }
I20260808 23:04:15.400208 27266 sys_catalog.cc:458] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:04:15.406631 27290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d02a10f30e743a5bdb328bc6072fc93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a10f30e743a5bdb328bc6072fc93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } } peers { permanent_uuid: "f198d1f307a24402a30ba695b9d7142e" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } } peers { permanent_uuid: "d33dbeb2564646bb8fc9d5d262f643c7" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } } }
I20260808 23:04:15.406718 27290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [sys.catalog]: This master's current role is: LEADER
I20260808 23:04:15.406772 27266 sys_catalog.cc:455] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d02a10f30e743a5bdb328bc6072fc93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a10f30e743a5bdb328bc6072fc93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } } peers { permanent_uuid: "f198d1f307a24402a30ba695b9d7142e" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } } peers { permanent_uuid: "d33dbeb2564646bb8fc9d5d262f643c7" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } } }
I20260808 23:04:15.406842 27266 sys_catalog.cc:458] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:04:15.406986 27322 mvcc.cc:205] Tried to move back new op lower bound from 7316399126108311552 to 7316399126099660800. Current Snapshot: MvccSnapshot[applied={T|T < 7316399126108311552}]
I20260808 23:04:15.407282 27327 mvcc.cc:205] Tried to move back new op lower bound from 7316399126108311552 to 7316399126099660800. Current Snapshot: MvccSnapshot[applied={T|T < 7316399126108311552}]
I20260808 23:04:15.408051 27027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.397912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47848 (local address 127.23.194.62:42245)
0808 23:04:15.397957 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.397959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.400559 (+ 2600us) server_negotiation.cc:407] Connection header received
0808 23:04:15.400654 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.400655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.400666 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.400695 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.401444 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.402160 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.403308 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.403591 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.403631 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.405157 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:15.405159 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:15.405159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:15.405166 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:15.406691 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.407176 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.407177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.407178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.407234 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.407422 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.407423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.407424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.407541 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:15.407631 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.407876 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.407950 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 23:04:15.410548 27321 catalog_manager.cc:1347] Generated new cluster ID: e8e2876acf74438f85be64bb20fcbc76
I20260808 23:04:15.410597 27321 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:04:15.410593 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.397669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47624 (local address 127.23.194.60:42917)
0808 23:04:15.397712 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.397714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.398721 (+ 1007us) server_negotiation.cc:407] Connection header received
0808 23:04:15.398945 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.398946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.398958 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.399001 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.400422 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.401162 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.402996 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.403315 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.403354 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.407227 (+ 3873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:15.407228 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:15.407229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:15.407236 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:15.409152 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.409801 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.409802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.409802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.409861 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.410058 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.410059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.410059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.410180 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:15.410251 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.410428 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.410501 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 23:04:15.413532 27133 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } attempt: 3
I20260808 23:04:15.418210 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.398215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50306 (local address 127.23.194.61:41227)
0808 23:04:15.398559 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.398561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.408865 (+ 10304us) server_negotiation.cc:407] Connection header received
0808 23:04:15.408957 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.408958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.408969 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.408998 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.410974 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.411682 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.412646 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.412963 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.413011 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.414909 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:15.414911 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:15.414911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:15.414919 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:15.416585 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.417259 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.417261 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.417261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.417326 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.417592 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.417593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.417593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.417728 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:15.417828 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.418026 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.418110 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315}
Aug 08 23:04:15 dist-test-slave-f0tn krb5kdc[26897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786230255, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.194.61@KRBTEST.COM for kudu/127.23.194.60@KRBTEST.COM
I20260808 23:04:15.422281 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.413764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.194.60:42917 (local address 127.0.0.1:47636)
0808 23:04:15.413844 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 23:04:15.413853 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 23:04:15.413887 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:04:15.414121 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:04:15.414122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:04:15.414302 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:04:15.414342 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:15.414344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.415283 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.415284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:15.416063 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:04:15.416065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.416578 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.416579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:04:15.416651 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.417626 (+ 975us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:04:15.417632 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:04:15.419235 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:04:15.421117 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:15.421118 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:15.421118 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:15.421332 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:15.421573 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:04:15.421574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:04:15.421574 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:04:15.421642 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:04:15.421963 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:04:15.421964 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:04:15.422029 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 23:04:15.422094 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 23:04:15.422196 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 23:04:15.422753 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.413948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47636 (local address 127.23.194.60:42917)
0808 23:04:15.413981 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.413983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.413988 (+ 5us) server_negotiation.cc:407] Connection header received
0808 23:04:15.414005 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.414005 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.414025 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.414067 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.414478 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.415178 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.416168 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.416461 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.416513 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.419336 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:04:15.419337 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:04:15.419337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:04:15.419344 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:04:15.420958 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.421431 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.421432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.421432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.421499 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:04:15.421711 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:04:15.421712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:04:15.421713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:04:15.421868 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:04:15.421925 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.422580 (+ 655us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.422666 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 23:04:15.424302 27133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f198d1f307a24402a30ba695b9d7142e: Bootstrap starting.
I20260808 23:04:15.425057 27133 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f198d1f307a24402a30ba695b9d7142e: Neither blocks nor log segments found. Creating new log.
I20260808 23:04:15.425367 27133 log.cc:824] T 00000000000000000000000000000000 P f198d1f307a24402a30ba695b9d7142e: Log is configured to *not* fsync() on all Append() calls
I20260808 23:04:15.427254 27133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f198d1f307a24402a30ba695b9d7142e: No bootstrap required, opened a new log
I20260808 23:04:15.432374 27133 raft_consensus.cc:348] T 00000000000000000000000000000000 P f198d1f307a24402a30ba695b9d7142e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a10f30e743a5bdb328bc6072fc93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } } peers { permanent_uuid: "f198d1f307a24402a30ba695b9d7142e" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } } peers { permanent_uuid: "d33dbeb2564646bb8fc9d5d262f643c7" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } }
I20260808 23:04:15.432730 27133 raft_consensus.cc:729] T 00000000000000000000000000000000 P f198d1f307a24402a30ba695b9d7142e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f198d1f307a24402a30ba695b9d7142e, State: Initialized, Role: FOLLOWER
I20260808 23:04:15.433032 27133 consensus_queue.cc:260] T 00000000000000000000000000000000 P f198d1f307a24402a30ba695b9d7142e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a10f30e743a5bdb328bc6072fc93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } } peers { permanent_uuid: "f198d1f307a24402a30ba695b9d7142e" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } } peers { permanent_uuid: "d33dbeb2564646bb8fc9d5d262f643c7" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } }
I20260808 23:04:15.433990 27095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d02a10f30e743a5bdb328bc6072fc93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f198d1f307a24402a30ba695b9d7142e"
I20260808 23:04:15.434022 27094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f198d1f307a24402a30ba695b9d7142e [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:04:15.435449 27094 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f198d1f307a24402a30ba695b9d7142e [term 1 FOLLOWER]: Refusing update from remote peer 3d02a10f30e743a5bdb328bc6072fc93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:04:15.435966 27310 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f198d1f307a24402a30ba695b9d7142e" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:04:15.437466 27095 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f198d1f307a24402a30ba695b9d7142e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d02a10f30e743a5bdb328bc6072fc93 in term 1.
I20260808 23:04:15.433900 27133 sys_catalog.cc:564] T 00000000000000000000000000000000 P f198d1f307a24402a30ba695b9d7142e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:04:15.444293 27321 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:04:15.445099 27321 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:04:15.445475 27335 sys_catalog.cc:455] T 00000000000000000000000000000000 P f198d1f307a24402a30ba695b9d7142e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d02a10f30e743a5bdb328bc6072fc93. Latest consensus state: current_term: 1 leader_uuid: "3d02a10f30e743a5bdb328bc6072fc93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a10f30e743a5bdb328bc6072fc93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } } peers { permanent_uuid: "f198d1f307a24402a30ba695b9d7142e" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } } peers { permanent_uuid: "d33dbeb2564646bb8fc9d5d262f643c7" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } } }
I20260808 23:04:15.445631 27335 sys_catalog.cc:458] T 00000000000000000000000000000000 P f198d1f307a24402a30ba695b9d7142e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:04:15.445710 27335 sys_catalog.cc:455] T 00000000000000000000000000000000 P f198d1f307a24402a30ba695b9d7142e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d02a10f30e743a5bdb328bc6072fc93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a10f30e743a5bdb328bc6072fc93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } } peers { permanent_uuid: "f198d1f307a24402a30ba695b9d7142e" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } } peers { permanent_uuid: "d33dbeb2564646bb8fc9d5d262f643c7" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } } }
I20260808 23:04:15.445779 27335 sys_catalog.cc:458] T 00000000000000000000000000000000 P f198d1f307a24402a30ba695b9d7142e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:04:15.446344 27339 mvcc.cc:205] Tried to move back new op lower bound from 7316399126108311552 to 7316399126099660800. Current Snapshot: MvccSnapshot[applied={T|T < 7316399126108311552}]
I20260808 23:04:15.447554 27290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d02a10f30e743a5bdb328bc6072fc93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a10f30e743a5bdb328bc6072fc93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } } peers { permanent_uuid: "f198d1f307a24402a30ba695b9d7142e" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } } peers { permanent_uuid: "d33dbeb2564646bb8fc9d5d262f643c7" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } } }
I20260808 23:04:15.447654 27290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93 [sys.catalog]: This master's current role is: LEADER
I20260808 23:04:15.442996 27334 sys_catalog.cc:455] T 00000000000000000000000000000000 P f198d1f307a24402a30ba695b9d7142e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d02a10f30e743a5bdb328bc6072fc93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a10f30e743a5bdb328bc6072fc93" member_type: VOTER last_known_addr { host: "127.23.194.62" port: 42245 } } peers { permanent_uuid: "f198d1f307a24402a30ba695b9d7142e" member_type: VOTER last_known_addr { host: "127.23.194.61" port: 41227 } } peers { permanent_uuid: "d33dbeb2564646bb8fc9d5d262f643c7" member_type: VOTER last_known_addr { host: "127.23.194.60" port: 42917 } } }
I20260808 23:04:15.448019 27334 sys_catalog.cc:458] T 00000000000000000000000000000000 P f198d1f307a24402a30ba695b9d7142e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:04:15.454810 27321 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d02a10f30e743a5bdb328bc6072fc93: Generated new TSK 0
I20260808 23:04:15.455139 27321 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:04:15.456036 27359 catalog_manager.cc:1259] Loaded cluster ID: e8e2876acf74438f85be64bb20fcbc76
I20260808 23:04:15.456095 27359 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f198d1f307a24402a30ba695b9d7142e: loading cluster ID for follower catalog manager: success
I20260808 23:04:15.457340 27359 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f198d1f307a24402a30ba695b9d7142e: acquiring CA information for follower catalog manager: success
I20260808 23:04:15.457777 27359 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f198d1f307a24402a30ba695b9d7142e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 23:04:15.486186 27262 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:15.484064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47642 (local address 127.23.194.60:42917)
0808 23:04:15.484116 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.484118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.485252 (+ 1134us) server_negotiation.cc:407] Connection header received
0808 23:04:15.485412 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.485413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.485428 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.485465 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.486114 (+ 649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47642: BlockingRecv error: recv got EOF from 127.0.0.1:47642 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260808 23:04:15.487254 27027 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:15.484537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47850 (local address 127.23.194.62:42245)
0808 23:04:15.484583 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.484585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.486589 (+ 2004us) server_negotiation.cc:407] Connection header received
0808 23:04:15.486692 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.486693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.486705 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.486735 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.487191 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47850: BlockingRecv error: recv got EOF from 127.0.0.1:47850 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 23:04:15.488060 27143 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:15.484912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50316 (local address 127.23.194.61:41227)
0808 23:04:15.484959 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.484961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.486862 (+ 1901us) server_negotiation.cc:407] Connection header received
0808 23:04:15.487570 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.487571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.487582 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.487611 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.487969 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50316: BlockingRecv error: recv got EOF from 127.0.0.1:50316 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 23:04:15.493753 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.489958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50332 (local address 127.23.194.61:41227)
0808 23:04:15.489994 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.489995 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.490204 (+ 209us) server_negotiation.cc:407] Connection header received
0808 23:04:15.490302 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.490303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.490344 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.490385 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:15.490588 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.491358 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.492679 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.492991 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.493035 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.493308 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:15.493472 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:15.493540 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.493657 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.493679 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 23:04:15.497223 27262 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:15.496336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47658 (local address 127.23.194.60:42917)
0808 23:04:15.496374 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.496376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.496625 (+ 249us) server_negotiation.cc:407] Connection header received
0808 23:04:15.496729 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.496730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.496750 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.496779 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.497156 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47658: BlockingRecv error: recv got EOF from 127.0.0.1:47658 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 23:04:15.502820 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.499624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50342 (local address 127.23.194.61:41227)
0808 23:04:15.499658 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.499660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.499895 (+ 235us) server_negotiation.cc:407] Connection header received
0808 23:04:15.499983 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.499984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.499993 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.500018 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:15.500178 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.500857 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.502034 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.502303 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.502347 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.502565 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:15.502600 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:15.502628 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.502733 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.502753 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 23:04:15.506117 27262 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:15.505297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47674 (local address 127.23.194.60:42917)
0808 23:04:15.505331 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.505333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.505606 (+ 273us) server_negotiation.cc:407] Connection header received
0808 23:04:15.505702 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.505703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.505715 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.505744 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.506072 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47674: BlockingRecv error: recv got EOF from 127.0.0.1:47674 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 23:04:15.512670 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.509368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50354 (local address 127.23.194.61:41227)
0808 23:04:15.509448 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.509449 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.509706 (+ 257us) server_negotiation.cc:407] Connection header received
0808 23:04:15.509860 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.509861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.509870 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.509897 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:15.510065 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.510752 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.511872 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.512151 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.512195 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.512394 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:15.512426 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:15.512454 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.512576 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.512598 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 23:04:15.515791 27262 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:15.514981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47678 (local address 127.23.194.60:42917)
0808 23:04:15.515017 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.515019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.515245 (+ 226us) server_negotiation.cc:407] Connection header received
0808 23:04:15.515339 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.515340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.515350 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.515379 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.515740 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47678: BlockingRecv error: recv got EOF from 127.0.0.1:47678 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 23:04:15.524369 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.521068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50370 (local address 127.23.194.61:41227)
0808 23:04:15.521102 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.521103 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.521410 (+ 307us) server_negotiation.cc:407] Connection header received
0808 23:04:15.521536 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.521537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.521546 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.521574 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:15.521774 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.522497 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.523611 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.523882 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.523924 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.524122 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:15.524156 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:15.524182 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.524282 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.524300 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 23:04:15.527379 27262 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:15.526656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47682 (local address 127.23.194.60:42917)
0808 23:04:15.526691 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.526693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.526887 (+ 194us) server_negotiation.cc:407] Connection header received
0808 23:04:15.526977 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.526978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.526989 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.527015 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.527332 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47682: BlockingRecv error: recv got EOF from 127.0.0.1:47682 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 23:04:15.539595 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.536547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50376 (local address 127.23.194.61:41227)
0808 23:04:15.536585 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.536587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.536822 (+ 235us) server_negotiation.cc:407] Connection header received
0808 23:04:15.536914 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.536915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.536925 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.536951 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:15.537117 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.537828 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.538839 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.539098 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.539139 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.539334 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:15.539366 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:15.539392 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.539491 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.539520 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 23:04:15.543022 27262 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:15.542143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47698 (local address 127.23.194.60:42917)
0808 23:04:15.542182 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.542184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.542385 (+ 201us) server_negotiation.cc:407] Connection header received
0808 23:04:15.542478 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.542479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.542490 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.542532 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.542974 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47698: BlockingRecv error: recv got EOF from 127.0.0.1:47698 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 23:04:15.563663 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.560337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50390 (local address 127.23.194.61:41227)
0808 23:04:15.560379 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.560381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.560638 (+ 257us) server_negotiation.cc:407] Connection header received
0808 23:04:15.560732 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.560734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.560745 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.560771 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:15.560931 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.561714 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.562827 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.563124 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.563161 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.563375 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:15.563424 (+ 49us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:15.563448 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.563559 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.563578 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 23:04:15.566841 27262 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:15.566107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47714 (local address 127.23.194.60:42917)
0808 23:04:15.566142 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.566144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.566343 (+ 199us) server_negotiation.cc:407] Connection header received
0808 23:04:15.566424 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.566425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.566435 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.566461 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.566792 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47714: BlockingRecv error: recv got EOF from 127.0.0.1:47714 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 23:04:15.602943 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.600156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50402 (local address 127.23.194.61:41227)
0808 23:04:15.600228 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.600231 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.600353 (+ 122us) server_negotiation.cc:407] Connection header received
0808 23:04:15.600416 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.600417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.600427 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.600476 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:15.600603 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.601085 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.602206 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.602389 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.602441 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.602638 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:15.602672 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:15.602749 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.602840 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.602861 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 23:04:15.605288 27262 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:15.604652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47728 (local address 127.23.194.60:42917)
0808 23:04:15.604701 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.604703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.604804 (+ 101us) server_negotiation.cc:407] Connection header received
0808 23:04:15.604848 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.604849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.604861 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.604908 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.605227 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47728: BlockingRecv error: recv got EOF from 127.0.0.1:47728 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 23:04:15.672669 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.670315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50414 (local address 127.23.194.61:41227)
0808 23:04:15.670383 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.670385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.670468 (+ 83us) server_negotiation.cc:407] Connection header received
0808 23:04:15.670540 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.670541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.670554 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.670603 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:15.670741 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.671229 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.672004 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.672225 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.672282 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.672411 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:15.672441 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:15.672481 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.672546 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.672566 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 23:04:15.674738 27262 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:15.674199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47730 (local address 127.23.194.60:42917)
0808 23:04:15.674241 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.674243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.674339 (+ 96us) server_negotiation.cc:407] Connection header received
0808 23:04:15.674380 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.674381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.674390 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.674432 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.674684 (+ 252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47730: BlockingRecv error: recv got EOF from 127.0.0.1:47730 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 23:04:15.806129 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:15.803775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50428 (local address 127.23.194.61:41227)
0808 23:04:15.803864 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.803867 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.804043 (+ 176us) server_negotiation.cc:407] Connection header received
0808 23:04:15.804116 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.804118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.804130 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.804200 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:15.804384 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.804895 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.805644 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:15.805821 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:15.805865 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:15.805959 (+ 94us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:15.805985 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:15.806016 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:15.806046 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 23:04:15.806060 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 23:04:15.808197 27262 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:15.807621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47732 (local address 127.23.194.60:42917)
0808 23:04:15.807689 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:15.807691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:15.807728 (+ 37us) server_negotiation.cc:407] Connection header received
0808 23:04:15.807788 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:15.807789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:15.807801 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:15.807842 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:15.808142 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47732: BlockingRecv error: recv got EOF from 127.0.0.1:47732 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 23:04:16.067893 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:16.065453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50434 (local address 127.23.194.61:41227)
0808 23:04:16.065513 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:16.065515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:16.065597 (+ 82us) server_negotiation.cc:407] Connection header received
0808 23:04:16.065666 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:16.065667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:16.065682 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:16.065735 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:16.065888 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:16.066399 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:16.067207 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:16.067401 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:16.067467 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:16.067638 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:16.067673 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:16.067723 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:16.067784 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 23:04:16.067807 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260808 23:04:16.070287 27262 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:04:16.069643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47744 (local address 127.23.194.60:42917)
0808 23:04:16.069713 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:16.069715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:16.069725 (+ 10us) server_negotiation.cc:407] Connection header received
0808 23:04:16.069786 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:16.069787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:16.069796 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:16.069852 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:04:16.070235 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47744: BlockingRecv error: recv got EOF from 127.0.0.1:47744 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 23:04:16.373744 27286 catalog_manager.cc:1259] Loaded cluster ID: e8e2876acf74438f85be64bb20fcbc76
I20260808 23:04:16.373812 27286 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7: loading cluster ID for follower catalog manager: success
I20260808 23:04:16.374583 27286 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7: acquiring CA information for follower catalog manager: success
I20260808 23:04:16.374832 27286 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d33dbeb2564646bb8fc9d5d262f643c7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 23:04:16.585696 27548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:16.583388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50436 (local address 127.23.194.61:41227)
0808 23:04:16.583514 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:16.583516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:16.583527 (+ 11us) server_negotiation.cc:407] Connection header received
0808 23:04:16.583575 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:16.583575 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:16.583587 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:16.583633 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:16.583744 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:16.584280 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:16.585029 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:16.585232 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:16.585283 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:16.585425 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:16.585464 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:16.585512 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:16.585563 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 23:04:16.585586 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":65,"threads_started":1}
I20260808 23:04:16.589284 27554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:16.587218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47758 (local address 127.23.194.60:42917)
0808 23:04:16.587334 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:16.587336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:16.587344 (+ 8us) server_negotiation.cc:407] Connection header received
0808 23:04:16.587366 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:16.587367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:16.587414 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:16.587479 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:16.587563 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:16.588027 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:16.588679 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:16.588861 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:16.588905 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:16.589003 (+ 98us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:16.589106 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:16.589159 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:16.589194 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 23:04:16.589210 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260808 23:04:16.593528 27554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:16.590966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47774 (local address 127.23.194.60:42917)
0808 23:04:16.591015 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:16.591017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:16.591108 (+ 91us) server_negotiation.cc:407] Connection header received
0808 23:04:16.591197 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:16.591198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:16.591208 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:16.591255 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:16.591350 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:16.592062 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:16.592869 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:16.593055 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:16.593107 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:16.593285 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:16.593316 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:16.593354 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:16.593439 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 23:04:16.593459 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 23:04:16.593936 27548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:16.590973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50442 (local address 127.23.194.61:41227)
0808 23:04:16.591022 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:16.591024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:16.591215 (+ 191us) server_negotiation.cc:407] Connection header received
0808 23:04:16.591310 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:16.591311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:16.591323 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:16.591369 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:16.591497 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:16.592061 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:16.593179 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:16.593505 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:16.593569 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:16.593701 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:16.593735 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:16.593760 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:16.593860 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 23:04:16.593884 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 23:04:16.594540 27563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:04:16.590833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47852 (local address 127.23.194.62:42245)
0808 23:04:16.591050 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 23:04:16.591052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 23:04:16.591061 (+ 9us) server_negotiation.cc:407] Connection header received
0808 23:04:16.591108 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:04:16.591109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:04:16.591167 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:04:16.591398 (+ 231us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:04:16.591438 (+ 40us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:16.592242 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:16.593650 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:04:16.593949 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:04:16.594018 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:04:16.594221 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:04:16.594326 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:04:16.594388 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 23:04:16.594461 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 23:04:16.594482 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":78,"threads_started":1}
I20260808 23:04:16.595968 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 26917
I20260808 23:04:16.610489 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 27034
I20260808 23:04:16.620441 24328 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taski6UDxc/build/release/bin/kudu with pid 27149
2026-08-08T23:04:16Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2952 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5872 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (275415 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST